httpd-proxy-scoreboard
Hi, I am trying to get the slotmem persistent (because for the moment it didn't persist even a graceful restart). So I have a question: - Is there a way to have a pool in a module that will only be cleaned up when httpd stops? Anyway I would like to have an option to have the slotmem persistent over restart of httpd or machine. In the proxy_health_checher a part of the slot mem is configuration information, that part only needs to be restored from the saved information only if the httpd.conf has not been changed. For the moment I will write the slot mems on disk and restore them. Probably that has to be configurable: like restoring always, only if configuration hasn't changed etc. Any comments? Cheers Jean-Frederic
Re: httpd-proxy-scoreboard
Mladen Turk wrote: Jean-frederic Clere wrote: Hi, I am trying to get the slotmem persistent (because for the moment it didn't persist even a graceful restart). So I have a question: - Is there a way to have a pool in a module that will only be cleaned up when httpd stops? I suppose you should do that like its done for scoreboard, by adding a hook to core.c ap_hook_pre_mpm(ap_create_scoreboard, NULL, NULL, APR_HOOK_MIDDLE); Not easy: once I have prevented the cleanup I have to find back the shared memory because the module has been unloaded and reloaded. Cheers Jean-Frederic Regards, Mladen.
add_member creates 2 workers
Hi, add_member() creates 2 workers when the worker is not already existing (it creates an extra worker if the worker is already existing). That is because of ap_proxy_add_worker_to_balancer(): +++ runtime-id = proxy_lb_workers; /* Increase the total runtime count */ proxy_lb_workers++; +++ I think that is wrong: The balancer does not need a worker it just needs a reference to a worker. Comments? Cheers Jean-Frederic
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Ruediger Pluem wrote:
On 07/28/2006 06:34 PM, [EMAIL PROTECTED] wrote:
Added:
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c?rev=426604view=auto
==
---
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c
(added)
+++
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c
Fri Jul 28 09:33:58 2006
+/* read the entry stored in the shared area and build the corresponding worker
structure */
+static apr_status_t get_entry(int id, proxy_worker **worker, char
**balancer_name, apr_pool_t *pool)
+{
+struct proxy_worker_conf *workerconf = NULL;
+apr_status_t rv;
+
+if (myscore == NULL)
+return APR_ENOSHMAVAIL;
+rv = checkstorage-ap_slotmem_mem(myscore, id, (void *) workerconf);
What should be done if we never have added an worker to this id and thus all
fields
in **workerconf are zero? Wouldn't it make sense to set *worker to NULL and
return
an error (something like NOT_INIT)?
I like the idea but the only error code I have found for it is APR_EINIT.
Any comments?
Cheers
Jean-Frederic
Regards
Rüdiger
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard:
Jim Jagielski wrote: Jean-frederic Clere wrote: Jim Jagielski wrote: A quick check shows that various worker stats are not shared... doing a reload of the balancer-manager shows the I/O/Elected values flopping all over the place. So they seem in this impl process specific and not shared at all. Did nothing special to build modules/mem just allowed default config and build... The shared is not create because $APACHE_HOME/proxy does not exist... And this is documented where? No doubt, if we're mapping shared mem to a actual file system location, it needs to be change-able or anonymous (in which case APR does it for us) Easy I have added a parameter (ProxySlotMemLoc) in mod_proxy. When using a value like: ProxySlotMemLoc proxy/comarea The shared memory associated file will be $ServerRoot/proxy/comarea. When using nothing the shared memory will anonymous. The internal name in this case is :mod_proxy.c. Comments? In fact I am not 100 % happy with it. Probably something like: ProxySlotMemLoc shm:proxy/comarea Would use shared memory to store the shared part of the workers. ProxySlotMemLoc plain:proxy/comarea Would use plain memory. Comments? Cheers Jean-Frederic
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard:
Jim Jagielski wrote: A quick check shows that various worker stats are not shared... doing a reload of the balancer-manager shows the I/O/Elected values flopping all over the place. So they seem in this impl process specific and not shared at all. Did nothing special to build modules/mem just allowed default config and build... The shared is not create because $APACHE_HOME/proxy does not exist... I'll try to look more over the weekend or next week, but I want to make sure to get my cluster-set patch in trunk soonish and start proposing some backports to 2.2.x :)
ordering pre_config hooks
Hi, I am trying to order pre_config hooks (using trunk) but that seem not to work. I have tried LAST/FIRST and the name but only the order in the config file seems to help. Any hints? Cheers Jean-Frederic
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard:
Jim Jagielski wrote: A quick check shows that various worker stats are not shared... doing a reload of the balancer-manager shows the I/O/Elected values flopping all over the place. So they seem in this impl process specific and not shared at all. Did nothing special to build modules/mem just allowed default config and build... I think I have fixed it... the directory of the filename to create the shared was probably missing. Cheers Jean-Frederic I'll try to look more over the weekend or next week, but I want to make sure to get my cluster-set patch in trunk soonish and start proposing some backports to 2.2.x :)
Re: svn commit: r426781 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/mem/sharedmem_util.c support/proxymonitor.c
Jim Jagielski wrote: I'm not sure I'm comfortable with the directory location (proxy) Ok Should it be moved in logs: The pid file is already there. and the names (checker|comarea) being hardcoded... Ok. That means I should not create the slot mem in the pre-config. And creating the directory is also somewhat unsettling as well since it's a normal directory that should exist. It would be like Apache pre-creating the logs directory... :/
Re: [Vote] product name - for shipping versions
Steffen wrote: I opt for: Apache Web Server No: Tomcat is also an Apache web server (via its DefaultServlet). Cheers Jean-Frederric Apache is more then only based on the HTTP protocol/specification , other official protocols/specifcations are used by Apache , eg. TLS, zlib Steffen - Original Message - From: William A. Rowe, Jr. [EMAIL PROTECTED] To: dev@httpd.apache.org Sent: Saturday, July 29, 2006 21:54 Subject: Re: [Vote] product name - for shipping versions William A. Rowe, Jr. wrote: [ ] httpd [ ] Apache httpd [ ] Apache httpd Server [X] Apache HTTP Server [ ] Other:
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Hi, I have committed the code to get comments on some points: - Does it make sense to include from support objects from modules/proxy? - Does the mod_proxy_health_checker is the right way to go? - I mean: one part is storing the worker information to use it in an external process and the other is the health check of a worker, should the module be cut in 2 pieces? - - Any other comments? Cheers Jean-Frederic On 7/28/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Author: jfclereDate: Fri Jul 28 09:33:58 2006New Revision: 426604URL: http://svn.apache.org/viewvc?rev=426604view=revLog:First try to put togother an external health checker for mod_proxy.Added:httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.chttpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.hhttpd/httpd/branches/httpd-proxy-scoreboard/support/proxymonitor.c Modified:httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy.chttpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy.h httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/proxy_util.chttpd/httpd/branches/httpd-proxy-scoreboard/support/Makefile.in+++ CUT +++
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Hi, I have committed the code to get comments on some points: - Does it make sense to include from support objects from modules/proxy? - Does the mod_proxy_health_checker is the right to do? - I mean: one part is storing the worker information to use it in an external process and the other is the health check of a worker, should the module be cut in 2 pieces? - - Any other comments? Cheers Jean-Frederic [EMAIL PROTECTED] wrote: Author: jfclere Date: Fri Jul 28 09:33:58 2006 New Revision: 426604 URL: http://svn.apache.org/viewvc?rev=426604view=rev Log: First try to put togother an external health checker for mod_proxy. Added: httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/health_checker_util.c httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.c httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.h httpd/httpd/branches/httpd-proxy-scoreboard/support/proxymonitor.c Modified: httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4 httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy.c httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy.h httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/proxy_util.c httpd/httpd/branches/httpd-proxy-scoreboard/support/Makefile.in +++ CUT +++
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Jim Jagielski wrote: On Jul 28, 2006, at 1:54 PM, jean-frederic clere wrote: Hi, I have committed the code to get comments on some points: - Does it make sense to include from support objects from modules/ proxy? - Does the mod_proxy_health_checker is the right way to go? - I mean: one part is storing the worker information to use it in an external process and the other is the health check of a worker, should the module be cut in 2 pieces? - - Any other comments? I thought that this was about abstracting out scoreboard so that other modules could have scoreboard-like access without mucking around with the real scoreboard... The scoreboard abstract is in modules/mem. Cheers Jean-Frederic
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Jim Jagielski wrote: On Jul 28, 2006, at 12:34 PM, [EMAIL PROTECTED] wrote: Author: jfclere Date: Fri Jul 28 09:33:58 2006 New Revision: 426604 URL: http://svn.apache.org/viewvc?rev=426604view=rev Log: First try to put togother an external health checker for mod_proxy. Just coming back from OSCON, I haven't had a chance to take a deeper look but MAN that's a lot of code and data duplication. What's the goal for the code? What's the push for an external health checker? I see different reasons to have an external health checker: - It does not need a request to start. - It tests the back-end server not only the socket used to send the data. - It allows to have the information that a worker is down before send a request to it (prevent useless retries). Cheers Jean-Frederic
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Jim Jagielski wrote: Many compile warnings when compiling with maint-mode: Fixed. Thanks, Cheers Jean-Frederic mod_proxy.c: In function 'add_pass': mod_proxy.c:1176: warning: implicit declaration of function 'proxy_checkstorage_add_entry' mod_proxy.c: In function 'proxy_post_config': mod_proxy.c:1870: warning: implicit declaration of function 'proxy_create_comarea' mod_proxy.c: In function 'proxy_pre_config': mod_proxy.c:2014: warning: implicit declaration of function 'proxy_lookup_storage_provider' In file included from proxy_util.c:20: mod_proxy_health_checker.h:32: warning: function declaration isn't a prototype proxy_util.c:2234: warning: no previous prototype for 'proxy_create_comarea' proxy_util.c:2244: warning: function declaration isn't a prototype proxy_util.c:2255: warning: no previous prototype for 'proxy_checkstorage_add_entry' In file included from mod_proxy_health_checker.c:15: mod_proxy_health_checker.h:32: warning: function declaration isn't a prototype mod_proxy_health_checker.c: In function 'healthck_pre_config': mod_proxy_health_checker.c:21: warning: implicit declaration of function 'health_checker_get_storage' mod_proxy_health_checker.c:21: warning: initialization makes pointer from integer without a cast mod_proxy_health_checker.c:26: warning: implicit declaration of function 'health_checker_init_slotmem_storage' mod_proxy_health_checker.c:30: warning: implicit declaration of function 'health_checker_init_slotmem' mod_proxy_health_checker.c: In function 'ap_healthstore_register_hook': mod_proxy_health_checker.c:57: warning: initialization makes pointer from integer without a cast mod_proxy_health_checker.c:54: warning: unused variable 'aszPre' In file included from health_checker_util.c:32: mod_proxy_health_checker.h:32: warning: function declaration isn't a prototype health_checker_util.c:129: warning: function declaration isn't a prototype health_checker_util.c: In function 'get_entry': health_checker_util.c:240: warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type health_checker_util.c:242: warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type health_checker_util.c:244: warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type health_checker_util.c:246: warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type health_checker_util.c:248: warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type health_checker_util.c: In function 'get_entryconf': health_checker_util.c:275: warning: passing argument 3 of 'checkstorage-ap_slotmem_mem' from incompatible pointer type health_checker_util.c: In function 'check_entryhealth': health_checker_util.c:289: warning: passing argument 3 of 'checkstorage-ap_slotmem_mem' from incompatible pointer type health_checker_util.c: At top level: health_checker_util.c:318: warning: initialization from incompatible pointer type health_checker_util.c:325: warning: function declaration isn't a prototype health_checker_util.c: In function 'health_checker_get_storage': health_checker_util.c:326: warning: return discards qualifiers from pointer target type health_checker_util.c: At top level: health_checker_util.c:331: warning: no previous prototype for 'health_checker_init_slotmem_storage' health_checker_util.c:335: warning: function declaration isn't a prototype health_checker_util.c: In function 'health_checker_get_slotmem_storage': health_checker_util.c:336: warning: return discards qualifiers from pointer target type health_checker_util.c: At top level: health_checker_util.c:341: warning: no previous prototype for 'health_checker_init_slotmem' health_checker_util.c:345: warning: function declaration isn't a prototype health_checker_util.c:306: warning: 'check_poolhealth' defined but not used sharedmem_util.c:52: warning: no previous prototype for 'cleanup_slotmem' sharedmem_util.c: In function 'ap_slotmem_create': sharedmem_util.c:139: warning: implicit declaration of function 'apr_pstrdup' sharedmem_util.c:139: warning: assignment makes pointer from integer without a cast sharedmem_util.c:83: warning: unused variable 'slotmem' sharedmem_util.c: In function 'ap_slotmem_attach': sharedmem_util.c:192: warning: assignment makes pointer from integer without a cast sharedmem_util.c:154: warning: unused variable 'slotmem' sharedmem_util.c: At top level: sharedmem_util.c:234: warning: function declaration isn't a prototype sharedmem_util.c: In function 'sharedmem_getstorage': sharedmem_util.c:235: warning: return discards qualifiers from pointer target type sharedmem_util.c: At top level: sharedmem_util.c:239: warning: no previous prototype for 'sharedmem_initglobalpool' sharedmem_util.c:244: warning: no previous prototype for 'sharedmem_initialize_cleanup'
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Davi Arnaut wrote:
Em 28/07/2006, às 13:34, [EMAIL PROTECTED] escreveu:
Author: jfclere
Date: Fri Jul 28 09:33:58 2006
New Revision: 426604
...
+
+static const slotmem_storage_method *checkstorage = NULL;
+static ap_slotmem_t *myscore=NULL;
Indentation consistency ? myscore=NULL
fixed.
+
+if (!port) {
+if (strcmp(scheme, ajp) == 0)
+port = 8009;
+else if (strcmp(scheme, http) == 0)
+port = 80;
+else
+port = 443;
+}
apr_uri_port_of_scheme ? (it may not have the ajp scheme, a patch is
appreciated)
Done.
+rv = apr_sockaddr_info_get(epsv_addr, hostname, APR_INET,
port, 0, pool);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ apr_sockaddr_info_get failed);
+apr_socket_close(newsock);
+return rv;
+}
ap_log_error(..APLOG_ERR, rv, NULL..) so we may have a hint why it
failed
Yep.
+rv = apr_socket_timeout_set(newsock, 10);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_WARNING, 0, NULL,
+apr_socket_timeout_set);
+apr_socket_close(newsock);
+return rv;
same for ap_log_error
+rv = apr_socket_connect(newsock, epsv_addr);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, NULL,
+apr_socket_connect failed);
+apr_socket_close(newsock);
+return rv;
+}
same for ap_log_error..and so on.
+
+if (balancer_name)
+strcpy(workerconf-balancer_name, balancer_name);
+workerconf-id = worker-id;
+workerconf-retry = worker-retry;
+workerconf-lbfactor = worker-lbfactor;
+if (worker-name)
+strcpy(workerconf-name, worker-name);
+if (worker-scheme)
+strcpy(workerconf-scheme, worker-scheme);
+if (worker-hostname)
+strcpy(workerconf-hostname, worker-hostname);
+if (worker-route)
+strcpy(workerconf-route, worker-route);
+if (worker-redirect)
+strcpy(workerconf-redirect, worker-redirect);
strncpy ?
Ok fixed.
+/* allocate the data */
+*worker = apr_pcalloc(pool, sizeof(proxy_worker));
+if (workerconf-balancer_name)
+*balancer_name = apr_pcalloc(pool, strlen(workerconf-
balancer_name));
+else
+*balancer_name = NULL;
allocated for what ? the string is not copied. Also, shoudn't it be
strlen(..) + 1 ?
Fixed.
+/* make the module usuable from outside */
+health_worker_method *health_checker_get_storage()
+{
+return(worker_storage);
+}
+
+/* handle the slotmem storage */
+void health_checker_init_slotmem_storage(slotmem_storage_method *
storage)
+{
+checkstorage = storage;
+}
+slotmem_storage_method * health_checker_get_slotmem_storage()
+{
+return(checkstorage);
+}
+
+/* handle the slotmen itself */
+void health_checker_init_slotmem(ap_slotmem_t *score)
+{
+ myscore = score;
+}
+ap_slotmem_t *health_checker_get_slotmem()
+{
+return(myscore);
+}
static APR_INLINE ...
No, why?
+char * ap_server_root_relative(apr_pool_t *p, const char *name)
+{
+char *fname;
+
+/* XXX: apr_filepath_merge better ? */
+if (basedir name[0] != '/') {
+fname = apr_pcalloc(p, strlen(basedir)+strlen(name)+1);
+strcpy(fname, basedir);
+strcat(fname, /);
+strcat(fname, name);
apr_pstrcat ?
Yes, done
Re: svn commit: r426604 - in /httpd/httpd/branches/httpd-proxy-scoreboard: modules/proxy/ support/
Ruediger Pluem wrote:
On 28.07.2006 18:34, [EMAIL PROTECTED] wrote:
Author: jfclere
Date: Fri Jul 28 09:33:58 2006
New Revision: 426604
URL: http://svn.apache.org/viewvc?rev=426604view=rev
Log:
First try to put togother an external health checker for mod_proxy.
Modified: httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4?rev=426604r1=426603r2=426604view=diff
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4
(original)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/config.m4 Fri Jul
28 09:33:58 2006
@@ -17,6 +17,7 @@
+}
+/* copy the worker information in the shared area so the health-checker can
extract the part it need */
+static apr_status_t add_entry(proxy_worker *worker, char *balancer_name, int
id)
What is the purpose of the id parameter. I do not see that it is used.
The id of the slot where the worker has to be stored.
+{
+struct proxy_worker_conf *workerconf = NULL;
+apr_status_t rv;
+
+if (myscore == NULL)
+return APR_ENOSHMAVAIL;
+rv = checkstorage-ap_slotmem_mem(myscore, worker-id, (void *)
workerconf);
+if (rv != APR_SUCCESS) {
+return rv;
+}
+
+if (balancer_name)
+strcpy(workerconf-balancer_name, balancer_name);
+workerconf-id = worker-id;
+workerconf-retry = worker-retry;
+workerconf-lbfactor = worker-lbfactor;
Is this approach thread safe / process safe or is there no need to be?
It should be used while parse the configuration or adding an entry...
Well it isn't my idea is to have the thread/process synchro stuff before
calling add_entry.
+if (worker-name)
+strcpy(workerconf-name, worker-name);
+if (worker-scheme)
+strcpy(workerconf-scheme, worker-scheme);
+if (worker-hostname)
+strcpy(workerconf-hostname, worker-hostname);
+if (worker-route)
+strcpy(workerconf-route, worker-route);
+if (worker-redirect)
+strcpy(workerconf-redirect, worker-redirect);
Don't you need to use strncpy here to avoid buffer overflows?
Of course.
+/* read the entry stored in the shared area and build the corresponding worker
structure */
+static apr_status_t get_entry(int id, proxy_worker **worker, char
**balancer_name, apr_pool_t *pool)
+{
+struct proxy_worker_conf *workerconf = NULL;
+apr_status_t rv;
+
+if (myscore == NULL)
+return APR_ENOSHMAVAIL;
+rv = checkstorage-ap_slotmem_mem(myscore, id, (void *) workerconf);
+if (rv != APR_SUCCESS)
+return rv;
+
+/* allocate the data */
+*worker = apr_pcalloc(pool, sizeof(proxy_worker));
+if (workerconf-balancer_name)
+*balancer_name = apr_pcalloc(pool, strlen(workerconf-balancer_name));
+else
+*balancer_name = NULL;
+
+/* The httpstatus is handle by httpd don't touch it here */
+(* worker)-id = workerconf-id;
+// XXX: what to do (* worker)-s = workerconf;
+(* worker)-retry = workerconf-retry;
+(* worker)-lbfactor = workerconf-lbfactor;
+if (workerconf-name)
+strcpy((* worker)-name, workerconf-name);
+if (workerconf-scheme)
+strcpy((* worker)-scheme, workerconf-scheme);
+if (workerconf-hostname)
+strcpy((* worker)-hostname, workerconf-hostname);
+if (workerconf-route)
+strcpy((* worker)-route, workerconf-route);
+if (workerconf-redirect)
+strcpy((* worker)-redirect, workerconf-redirect);
Don't you need to allocate space for this first (like with the balancer name)?
Yes, done.
Added:
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.c?rev=426604view=auto
==
---
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.c
(added)
+++
httpd/httpd/branches/httpd-proxy-scoreboard/modules/proxy/mod_proxy_health_checker.c
Fri Jul 28 09:33:58 2006
+
+static int healthck_pre_config(apr_pool_t *pconf, apr_pool_t *plog,
+ apr_pool_t *ptemp)
+{
+slotmem_storage_method *checkstorage;
+health_worker_method *worker_storage = health_checker_get_storage();
+ap_slotmem_t *myscore;
+
+checkstorage = ap_lookup_provider(SLOTMEM_STORAGE, shared, 0);
+if (checkstorage) {
+health_checker_init_slotmem_storage(checkstorage);
+}
+if (checkstorage worker_storage) {
+checkstorage-ap_slotmem_create(myscore, proxy/checker,
worker_storage-getentrysize(), 128, pconf);
+health_checker_init_slotmem(myscore);
+}
+return OK;
+}
+
+/* XXX: Was to
Re: httpd-proxy-scoreboard
Ruediger Pluem wrote: On 07/26/2006 10:53 PM, Jean-frederic Clere wrote: I already have a prototype of an external health checker process that uses an AJP cping/cpong and a simple connect for http/https. Basicaly mod_proxy uses the health_worker_method routine to write and read from a slot mem that contains the information of the workers + a health information. A balancer could check the health information before using the worker. The health checker process uses the worker description (thru the health_worker_method) to check if the back-end service is running. Sorry, you may have explained that before, but why using an external health checker process? The idea is to have the health check independant from the requests so that httpd knows in advance that a worker is not working. My thoughts are more the line to do a health check on a connection every time you lease it from the pool. That means you try a worker that may be down since a while (x seconds). If I use something request driven, tell check every x seconds I would like to check all workers, how to do that without slowing down the request that causes the health check? I will make a try ;-) Cheers Jean-Frederic Regards Rüdiger
Re: httpd-proxy-scoreboard
Ruediger Pluem wrote: On 07/27/2006 11:31 AM, Jean-frederic Clere wrote: Ruediger Pluem wrote: On 07/26/2006 10:53 PM, Jean-frederic Clere wrote: I already have a prototype of an external health checker process that uses an AJP cping/cpong and a simple connect for http/https. Basicaly mod_proxy uses the health_worker_method routine to write and read from a slot mem that contains the information of the workers + a health information. A balancer could check the health information before using the worker. The health checker process uses the worker description (thru the health_worker_method) to check if the back-end service is running. Sorry, you may have explained that before, but why using an external health checker process? The idea is to have the health check independant from the requests so that httpd knows in advance that a worker is not working. Ah, now I get it. Ok, I guess we both have slightly different ideas about what to health check. Your idea is more of a server health check and is somewhat similar to what HW loadbalancers are doing: Check each backend on a regular schedule if it is still available. Yep. My idea is more of a connection health check. I want to check if the connection I leased from the pool is still healthy. You want to check all the connections of the pool corresponding to the worker but not all the workers. If it is not that does not mean necessarily that the server in the backend failed. That probably means the socket has been closed ;-) It could be just this connection. So the default response on a faulty connection would be to close this one and try with a new one. With TC that means you create a new thread in TC, we must not overload the back-end server. Only if this fails also then there seems to be something wrong with the backend server. This is more like what most DB connection pools offer with a health-check query. I think both ideas make sense. Of course the connection health checks should be simpler than the server health check to avoid too much overhead and too much lost time. That is the reason why I think that e.g. doing a HTTP request on a HTTP connection leased from the pool is unreasonable. Yes... That is hard to check if http/https back-end server is healthy or not. We need some quick network layer check in this case, whereas in the AJP case CPING/CPONG has still an acceptable overhead. Finally I think if we have a good provider interface we can use the health check providers for both kind of health checks. Right an internal httpd provide would have the 2 health checks ways and external one probably one. If the health check is external we only need to process the result, if it is internal both health check and result processing are needed. We only need to create the frameworks for them to start in different parts of the code (separate process in your case, in my case I would guess that it needs to replace is_socket_connected which is called from ap_proxy_connect_backend) Regards Rüdiger
Re: svn commit: r425734 - in /httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem: config5.m4 mod_plainmem.c mod_scoreboard.c mod_sharedmem.c sharedmem_util.c slotmem.h
Ruediger Pluem wrote:
On 26.07.2006 15:42, [EMAIL PROTECTED] wrote:
Author: jfclere
Date: Wed Jul 26 06:42:43 2006
New Revision: 425734
URL: http://svn.apache.org/viewvc?rev=425734view=rev
Log:
Add ap_slotmem_attach() to the slotmem_storage_method.
Cut mod_sharemem.c in 2 so that its features could be
used outside httpd.
Modified: httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c?rev=425734r1=425733r2=425734view=diff
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c
(original)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c Wed
Jul 26 06:42:43 2006
@@ -98,6 +98,36 @@
*new = res;
return APR_SUCCESS;
}
+
+static apr_status_t ap_slotmem_attach(ap_slotmem_t **new, const char *name,
apr_size_t item_size, int item_num, apr_pool_t *pool)
Why using item_size and item_num as call by value parameters? Shouldn't that be
pointers?
Yep.
+{
+void *slotmem = NULL;
+ap_slotmem_t *res;
+ap_slotmem_t *next = globallistmem;
+char *fname;
+apr_status_t rv;
+
+fname = ap_server_root_relative(pool, name);
+
+/* first try to attach to existing slotmem */
+if (next) {
+for (;;) {
+if (strcmp(next-name, fname) == 0) {
+/* we already have it */
+*new = next;
+*item_size = next-size;
+*item_num = next-num;
See comment above.
Modified:
httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c?rev=425734r1=425733r2=425734view=diff
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
(original)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
Wed Jul 26 06:42:43 2006
@@ -84,6 +84,12 @@
*new = res;
return APR_SUCCESS;
}
+
+static apr_status_t ap_slotmem_attach(ap_slotmem_t **new, const char *name,
apr_size_t *item_size, int *item_num, apr_pool_t *pool)
+{
+return(ap_slotmem_create(new, name, item_size, item_num, pool));
This does not work. You need to provide ap_slotmem_create with values for
item_size and item_num. It does not return them. This can get difficult since
these values get not stored in the scoreboard. There is not globallistmem here.
Right... Well I have used proxy_lb_workers() for the number of slot mem.
I have the problem is other places to get proxy_lb_workers() in
pre-config. I don't how to make this clean for the moment.
Added: httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/sharedmem_util.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/sharedmem_util.c?rev=425734view=auto
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/sharedmem_util.c
(added)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/sharedmem_util.c
Wed Jul 26 06:42:43 2006
+static apr_status_t ap_slotmem_attach(ap_slotmem_t **new, const char *name,
apr_size_t *item_size, int *item_num, apr_pool_t *pool)
+{
+void *slotmem = NULL;
+void *ptr;
+ap_slotmem_t *res;
+ap_slotmem_t *next = globallistmem;
+struct sharedslotdesc desc;
+char *fname;
+apr_status_t rv;
+
+fname = ap_server_root_relative(pool, name);
+
+/* first try to attach to existing slotmem */
+if (next) {
+for (;;) {
+if (strcmp(next-name, fname) == 0) {
+/* we already have it */
+*new = next;
+*item_size = next-size;
+*item_num = next-num;
+return APR_SUCCESS;
+}
+if (next-next)
+break;
+next = next-next;
+}
+}
+
+/* first try to attach to existing shared memory */
+res = (ap_slotmem_t *) apr_pcalloc(globalpool, sizeof(ap_slotmem_t));
+rv = apr_shm_attach(res-shm, fname, globalpool);
+if (rv != APR_SUCCESS)
+return rv;
+
+/* Read the description of the slotmem */
+ptr = apr_shm_baseaddr_get(res-shm);
+memcpy(desc, ptr, sizeof(desc));
+ptr = ptr + sizeof(desc);
+
+/* For the chained slotmem stuff */
+res-name = apr_pstrdup(globalpool, fname);
+res-base = ptr;
+res-size = desc.item_size;
+res-num = desc.item_num;
+res-next = NULL;
+if (globallistmem==NULL)
+globallistmem = res;
+else
+next-next = res;
+
+*new = res;
+*item_size = desc.item_size;
+*item_num = desc.item_num;
+return APR_SUCCESS;
Too much copy and
httpd-proxy-scoreboard
Hi, I have started to write a generic health-checker for mod_proxy. I would like to change the macro PROXY_WORKER_IS_USABLE() to a routine in proxy_util.c. Comments? Another problem I have is to decide the max size of the slot mem for the worker: proxy_lb_workers() only gives the right value after parsing the configuration file (in the post-config()) but the logic creates the workers while parsing the configuration file. What to do? Create the slot mem with a fixed value (like 128: that is what I have done in my prototype)? Create the slot mem in plain memory and do the create of the real slot mem and create the workers in the post-config()? Cheers Jean-Frederoc
Re: httpd-proxy-scoreboard
Ruediger Pluem wrote:
On 26.07.2006 18:11, Jean-frederic Clere wrote:
Hi,
I have started to write a generic health-checker for mod_proxy. I
would like to change the macro PROXY_WORKER_IS_USABLE() to a routine in
proxy_util.c.
Comments?
From my current point of view we should keep this macro as a simple bit checker.
+1 to your efforts writing a generic health-checker to mod_proxy. Do you use
a provider approach to implement it?
Yes. I have attached a draft of the include I plan to use.
That would make it possible to reimplement
Mladen's cping_cpong for AJP as a provider.
I already have a prototype of an external health checker process that
uses an AJP cping/cpong and a simple connect for http/https.
Basicaly mod_proxy uses the health_worker_method routine to write and
read from a slot mem that contains the information of the workers + a
health information. A balancer could check the health information before
using the worker. The health checker process uses the worker description
(thru the health_worker_method) to check if the back-end service is running.
Then we would have a very flexible
way to plugin different health-checks.
Yep
Cheers
Jean-Frederic
Regards
Rüdiger
/* To store the configuration of the balancers and workers.
*/
struct proxy_balancer_conf {
char name[32];
char sticky[32];
int sticky_force;
apr_interval_time_t timeout;
int max_attempts;
char max_attempts_set;
char lbmethod_name[32];
};
struct proxy_worker_conf {
proxy_worker_stat httpstatus; /* httpd private */
char balancer_name[32];
int id;/* scoreboard id */
apr_interval_time_t retry; /* retry interval */
int lbfactor; /* initial load balancing factor */
charname[32];
charscheme[6]; /* scheme to use ajp|http|https */
charhostname[32]; /* remote backend address */
charroute[128];/* balancing route */
charredirect[128]; /* temporary balancing redirection route */
int status;/* temporary worker status */
apr_port_t port;
int min; /* Desired minimum number of available connections */
int smax; /* Soft maximum on the total number of connections */
int hmax; /* Hard maximum on the total number of connections */
apr_interval_time_t ttl; /* maximum amount of time in seconds a connection
* may be available while exceeding the soft limit */
apr_interval_time_t timeout; /* connection timeout */
chartimeout_set;
apr_interval_time_t acquire; /* acquire timeout when the maximum number of connections is exceeded */
characquire_set;
apr_size_t recv_buffer_size;
charrecv_buffer_size_set;
apr_size_t io_buffer_size;
chario_buffer_size_set;
charkeepalive;
charkeepalive_set;
int is_address_reusable;
int flush_packets;
int flush_wait; /* poll wait time in microseconds if flush_auto */
int health;
};
Re: Balancer manager: bug and feature request
Andrew Stribblehill wrote: That is, bug and feature request, not bug and feature request. As I reported in http://issues.apache.org/bugzilla/show_bug.cgi?id=39907 I'm finding that with the worker MPM and a pristine httpd 2.2.2 on an i386 Debian platform, changes I make to the configuration within balancer-manager will periodically reset -- and it's faster under load. I can see the code where the config is being set, but I don't quite know whether this is process-local or not. If so, this would explain the symptoms. The problem is that the information you entered is ONLY process-local. If someone could point me in the right direction, I'll have a bash at fixing this bug. The feature request: sometimes we need to take down an origin server on a planned basis. The app is session-based so we can't just disable it completely. Is there any reason I shouldn't be able to set the load factor for an origin server to 0? At present it's constrained to the range 1..100 inclusive. If I had some form of session replication, I could use route-redirect and then just disable it, but this is not possible for me. That won't work anyway due to the problem you describe above ;-( Thanks
Re: proxy balancer backports for 2.2.3
Jim Jagielski wrote: I don't want the 2.2.3 release to be held up too much, but there are some things being worked on in the proxy balancer code that would be useful to be available in 2.2.3. Mladen and JFC are working on the AJP stuff and as soon as they commit their changes, I'll be adding in my balancer set patches (to trunk). For me I am busy with the code for the branch httpd-proxy-scoreboard ;-) Cheers Jean-Frederic But if 2.2.3 is expected to be sooner than that, maybe the hot-standby is enough for now...
Re: svn commit: r423444 - in /httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem: ./ Makefile.in config5.m4 mod_plainmem.c mod_scoreboard.c mod_sharedmem.c slotmem.h
Brian Akins wrote: Jean-frederic Clere wrote: Do you mean the proxy back-end connections? I am thinking of a more general purpose slotmem not particularly tied to proxy. Maybe have some wrapper functions that create a slotmem based on threads x procs and can be access using r-connection. (internally slotmem could use r-connection-id). Yes the slotmen could be used for that. Cheers Jean-Frederic
Re: svn commit: r423444 - in /httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem: ./ Makefile.in config5.m4 mod_plainmem.c mod_scoreboard.c mod_sharedmem.c slotmem.h
Ruediger Pluem wrote:
Some comments inline.
Regards
Many thanks
Cheers
Jean-Frederic
Rüdiger
On 19.07.2006 14:18, [EMAIL PROTECTED] wrote:
Author: jfclere
Date: Wed Jul 19 05:18:10 2006
New Revision: 423444
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c
(added)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_plainmem.c Wed
Jul 19 05:18:10 2006
@@ -0,0 +1,145 @@
+/* Copyright 1999-2006 The Apache Software Foundation or its licensors, as
+ * applicable.
+ *
+ * Licensed under the Apache License, Version 2.0 (the License);
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
Please use the latest header approved by the board in all files.
Done.
+}
+static apr_status_t ap_slotmem_create(ap_slotmem_t **new, const char *name,
apr_size_t item_size, int item_num, apr_pool_t *pool)
+{
+void *slotmem = NULL;
+ap_slotmem_t *res;
+ap_slotmem_t *next = globallistmem;
+char *fname;
+apr_status_t rv;
+
+fname = ap_server_root_relative(pool, name);
+
+/* first try to attach to existing slotmem */
+if (next) {
+for (;;) {
+if (strcmp(next-name, fname) == 0) {
+/* we already have it */
+*new = next;
+return APR_SUCCESS;
+}
+if (next-next)
Shouldn't this be (!next-next)?
Yep.
+break;
+next = next-next;
+}
+}
+
+/* create the memory using the globalpool */
+res = (ap_slotmem_t *) apr_pcalloc(globalpool, sizeof(ap_slotmem_t));
+res-base = apr_pcalloc(globalpool, item_size * item_num);
+if (!res-base)
+return APR_ENOSHMAVAIL;
+memset(res-base, 0, item_size * item_num);
Is this needed? You did a calloc.
Argh too fast in reusing the shared memory code.
+
+/* For the chained slotmem stuff */
+res-name = apr_pstrdup(globalpool, fname);
+res-size = item_size;
+res-num = item_num;
+res-next = NULL;
+if (globallistmem==NULL)
+globallistmem = res;
+else
+next-next = res;
+
+*new = res;
+return APR_SUCCESS;
+}
+static apr_status_t ap_slotmem_mem(ap_slotmem_t *score, int id, void**mem)
+{
+
+void *ptr = score-base + score-size * id;
+
+if (!score)
+return APR_ENOSHMAVAIL;
Shouldn't this check happen before assigning a value to ptr?
Well after all the checks.
Added: httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c?rev=423444view=auto
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
(added)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_scoreboard.c
Wed Jul 19 05:18:10 2006
+static apr_status_t ap_slotmem_mem(ap_slotmem_t *score, int id, void**mem)
+{
+void *ptr;
+if (!score)
+return APR_ENOSHMAVAIL;
+
+#if PROXY_HAS_SCOREBOARD
+if (ap_scoreboard_image)
+ptr = (void *)ap_get_scoreboard_lb(id);
+#else
+return APR_ENOSHMAVAIL;
+#endif
+
+if (!ptr)
+return APR_ENOSHMAVAIL;
+*mem = ptr;
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ap_slotmem_mem: score %d, score);
Shouldn't this be APLOG_DEBUG?
I have removed the ap_log_error().
Added: httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_sharedmem.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_sharedmem.c?rev=423444view=auto
==
--- httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_sharedmem.c
(added)
+++ httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/mod_sharedmem.c Wed
Jul 19 05:18:10 2006
+static apr_status_t ap_slotmem_create(ap_slotmem_t **new, const char *name,
apr_size_t item_size, int item_num, apr_pool_t *pool)
+{
+void *slotmem = NULL;
+ap_slotmem_t *res;
+ap_slotmem_t *next = globallistmem;
+char *fname;
+apr_status_t rv;
+
+fname = ap_server_root_relative(pool, name);
+
+/* first try to attach to existing slotmem */
+if (next) {
+for (;;) {
+if (strcmp(next-name, fname) == 0) {
+/* we already have it */
+*new = next;
+return APR_SUCCESS;
+
Re: mod_proxy feature coming up
Jim Jagielski wrote: Pretty soon I'll be committing my balancer set patch to httpd-trunk. This basically allows for member sets within a balancer similar to the 'distance' mod_jk attribute. What it does is allow for more control over which members will be used via m_p_b. The logic is: Look for all usable members in set #0 If none exist, look for any host standbys in #0 If none exist, select the next set and loop I think this design provides the most flexibility for the admin. Anyway, this will require an update to the pertinent data struct to add the set element. I will also at this time likely add a busy element as well, in anticipation of the by-busyness LB method patch ;) Nice I have also prepared memory slot handler to replace the scoreboard space used by mod_proxy may I should wait before committing the part that affects mod_proxy.c, proxy_util.c and mod_proxy.h? Cheers Jean-Frederic
Re: mod_proxy feature coming up
Jim Jagielski wrote: Jean-frederic Clere wrote: Jim Jagielski wrote: Pretty soon I'll be committing my balancer set patch to httpd-trunk. This basically allows for member sets within a balancer similar to the 'distance' mod_jk attribute. What it does is allow for more control over which members will be used via m_p_b. The logic is: Look for all usable members in set #0 If none exist, look for any host standbys in #0 If none exist, select the next set and loop I think this design provides the most flexibility for the admin. Anyway, this will require an update to the pertinent data struct to add the set element. I will also at this time likely add a busy element as well, in anticipation of the by-busyness LB method patch ;) Nice I have also prepared memory slot handler to replace the scoreboard space used by mod_proxy may I should wait before committing the part that affects mod_proxy.c, proxy_util.c and mod_proxy.h? Yes, I'd propose waiting to commit that. The sole reason is that the member-set and other previously committed patches will likely be more readily approved for backporting to 2.2.x, whereas the scoreboard changes might be more difficult :) Ok. That gives me time to write more memory slot handler. Cheers Jean-Frederic
Re: mod_proxy feature coming up
Jim Jagielski wrote: On Jul 18, 2006, at 11:29 AM, Jim Jagielski wrote: Yes, I'd propose waiting to commit that. The sole reason is that the member-set and other previously committed patches will likely be more readily approved for backporting to 2.2.x, whereas the scoreboard changes might be more difficult :) Of course, another option would be creating a quick httpd-trunk branch (httpd-proxy-scoreboard??) and committing the patch there, and we can then merge at some soon future point. We should really use SVN's cheap branches as much as possible... Ok I will create a branch and we will merge it later. Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in
Jim Jagielski wrote: Jean-frederic Clere wrote: Using the result of your ideas and the explaintions now I have mod_proxy that uses the scoreboard via a scoreboard provider ;-) Find enclosed the code. Just one comment off the top of my head: this seems to confuse what is the actual scoreboard and what is the proxy's segment of scoreboard space... Eg the ap_scoreboard_* stuff implies that it's Apache's real scoreboard. Right I will look for a better name. Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in
Brian Akins wrote: Jim Jagielski wrote: at is the actual scoreboard and what is the proxy's segment of scoreboard space... Eg the ap_scoreboard_* stuff implies that it's Apache's real scoreboard. Like I said, I think this scoreboard stuff should be more generic than just proxy. It could be renamed to avoid confusion with the real scoreboard. But, there is no reason the real scoreboard couldn't use mod_scoreboard itself. It would just have to be core. probably best to have mod_scoreboard now, and merge the two later. I have named it slotmem because really could be used for anything that needs shared memory slots, now its code has nothing to with mod_proxy, should I commit it in a new modules subdirectory like modules/mem? Cheers Jean-Frederic
mem memory providers
Hi, I am starting an thread that could go under ap_proxy_get_worker and proxy_worker_stat in mod_proxy ;-) Because I would like to create a new family of modules under mem (httpd/modules/mem): shared memory provider. The idea is to have modules that provides memory that could used by slots (pieces). Why: - The proxy balancing shared area is in the scoreboard could be good to have it outside the scoreboard (BTW each actual proxy entry in scoreboard is 1024 bytes fixed size and mostly not used). - That allows to have part of this shared area RW for httpd and part RW for another process to do some external health check or more. - The scoreboard itself could use that memory. Of course the first module I have written is for mod_proxy and use the actual scoreboard ;-) Comments? Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in
Jim Jagielski wrote: Brian Akins wrote: Jim Jagielski wrote: +1. For example, a memcached based scoreboard would be pretty cool ;) maybe in mod_scoreboard it may use a provider mechanism to actually implement the scoreboard. Maybe have an ap_scoreboard_create_ex where you could explicitly name a provider. Yeah, that's what I was thinking as well! My idea was to have something that could accessed by a non-httpd process... And that doesn't need the actual scoreboard but right the something could replace scoreboard. Ok I will rethink the concept. Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in mod_proxy
Brian Akins wrote: Jim Jagielski wrote: If this is data that needs to be accessed from non-proxy modules then yes, I agree. A basic API could look like. By worker, I am thinking about the mpm sense, not the proxy sense. I guess slot may be a better term: /*used for ap_scoreboard_do. mem is the memory associated with a worker, data is what is passed to ap_scoreboard_do. pool is pool used to create scoreboard*/ typedef apr_status_t ap_scoreboard_callback_fn_t(void* mem, void *data, apr_pool_t *pool); /*call the callback on all worker slots*/ AP_DECLARE(apr_status_t)ap_scoreboard_do(ap_scoreboard_t *s, ap_scoreboard_callback_fn_t *func, void *data, apr_pool_t *pool); /*create a new scoreboard with each item size is item_size. name is a key used for debuggin and in mod_status output. This would create shared memory, basically*/ AP_DECLARE(apr_status_t) ap_scoreboard_create(ap_scoreboard_t **new, const char *name, apr_size_t item_size, apr_pool_t *pool); /*get the memory associated with this worker slot. use c-id or c-sbh to get offset into shared memory*/ AP_DECLARE(apr_status_t) ap_scoreboard_mem(ap_scoreboard_t *s, conn_rec *c, void**mem); Thoughts. Somthing very similar to this is used by several very busy web sites... With such an interface you assume only one process will access to one slot... That is what the scoreboard allows. Allowing updates from different proccesses on the same slot. Should we have an ap_slot_read_look() and an ap_slot_unlock() for that? Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in mod_proxy
Brian Akins wrote: Jean-frederic Clere wrote: With such an interface you assume only one process will access to one slot... That is what the scoreboard allows. Allowing updates from different proccesses on the same slot. Should we have an ap_slot_read_look() and an ap_slot_unlock() for that? No. we don't have such for the built-in scoreboard. Anything can read the scoreboard, only current worker slot can change it. that's why in the sample API, to get the memory you pass a conn_rec. If it's slow, people won't use it. Semaphores are generally slow. Enforcing it by convention like we currently do seems reasonable. The workers shared information is shared by the processes already, all the processes may access the same slot or I am complety wrong? Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in mod_proxy
Brian Akins wrote: Jim Jagielski wrote: Having some external (or even internal) process update a slot that isn't its own is dangerous. And the required locking would be slow. In my own hacked proxy, an external healthchecker and the proxy share a piece of shared memory that is read-only by apache and read-write by the external health check. This is not scoreboard info, just some health info. The 2 are separate things. Right I just have to make the slot little bigger, that is cheaper that having a semaphore for that. Cheers Jean-Frederic
Re: Additing a storage for the shared information of the worker in mod_proxy
Brian Akins wrote:
Jim Jagielski wrote:
If this is data that needs to be accessed from non-proxy modules
then yes, I agree.
A basic API could look like. By worker, I am thinking about the mpm
sense, not the proxy sense. I guess slot may be a better term:
/*used for ap_scoreboard_do. mem is the memory associated with a
worker, data is what is passed to ap_scoreboard_do. pool is pool used
to create scoreboard*/
typedef apr_status_t ap_scoreboard_callback_fn_t(void* mem, void
*data, apr_pool_t *pool);
/*call the callback on all worker slots*/
AP_DECLARE(apr_status_t)ap_scoreboard_do(ap_scoreboard_t *s,
ap_scoreboard_callback_fn_t *func, void *data, apr_pool_t *pool);
/*create a new scoreboard with each item size is item_size. name is a
key used for debuggin and in mod_status output. This would create
shared memory, basically*/
AP_DECLARE(apr_status_t) ap_scoreboard_create(ap_scoreboard_t **new,
const char *name, apr_size_t item_size, apr_pool_t *pool);
/*get the memory associated with this worker slot. use c-id or c-sbh
to get offset into shared memory*/
AP_DECLARE(apr_status_t) ap_scoreboard_mem(ap_scoreboard_t *s,
conn_rec *c, void**mem);
Thoughts. Somthing very similar to this is used by several very busy
web sites...
Using the result of your ideas and the explaintions now I have mod_proxy
that uses the scoreboard via a scoreboard provider ;-)
Find enclosed the code.
Any comments?
The next step is to write shared memory scoreboard provider.
Cheers
Jean-Frederic
Index: mod_proxy_store_score.c
===
--- mod_proxy_store_score.c (revision 0)
+++ mod_proxy_store_score.c (revision 0)
@@ -0,0 +1,102 @@
+/* Copyright 1999-2006 The Apache Software Foundation or its licensors, as
+ * applicable.
+ *
+ * Licensed under the Apache License, Version 2.0 (the License);
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* Comarea for the mod_proxy modules: This one uses the scoreboard */
+#define CORE_PRIVATE
+
+#include mod_proxy.h
+#include mod_proxy_store.h
+#include ap_mpm.h
+#include scoreboard.h
+
+struct ap_scoreboard {
+void *ptr;
+apr_size_t size;
+int num;
+};
+
+static apr_status_t ap_scoreboard_do(ap_scoreboard_t *s,
ap_scoreboard_callback_fn_t *func, void *data, apr_pool_t *pool)
+{
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ ap_scoreboard_do);
+return 0;
+}
+static apr_status_t ap_scoreboard_create(ap_scoreboard_t **new, const char
*name, apr_size_t item_size, int item_num, apr_pool_t *pool)
+{
+void *score = NULL;
+ap_scoreboard_t *res;
+
+#if PROXY_HAS_SCOREBOARD
+if (ap_scoreboard_image) {
+score = (void *)ap_get_scoreboard_lb(0);
+if (!score)
+return APR_ENOSHMAVAIL;
+}
+#endif
+if (!score)
+score = apr_pcalloc(pool, item_size * item_num);
+res = (ap_scoreboard_t *) apr_pcalloc(pool, sizeof(ap_scoreboard_t));
+res-ptr = score;
+res-size = item_size;
+res-num = item_num;
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ ap_scoreboard_create);
+*new = res;
+return APR_SUCCESS;
+}
+static apr_status_t ap_scoreboard_mem(ap_scoreboard_t *score, int id,
void**mem)
+{
+
+void *ptr = score-ptr + (score-size * id);
+#if PROXY_HAS_SCOREBOARD
+if (ap_scoreboard_image)
+score = (void *)ap_get_scoreboard_lb(id);
+#endif
+
+if (!score)
+return APR_ENOSHMAVAIL;
+*mem = ptr;
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ ap_scoreboard_mem);
+return APR_SUCCESS;
+}
+
+static const proxy_storage_method storage = {
+ap_scoreboard_do,
+ap_scoreboard_create,
+ap_scoreboard_mem
+};
+
+static void child_init(apr_pool_t *p, server_rec *s)
+{
+}
+
+static void ap_proxy_store_score_register_hook(apr_pool_t *p)
+{
+ap_hook_child_init(child_init, NULL, NULL, APR_HOOK_MIDDLE);
+ap_register_provider(p, PROXY_STORAGE, score, 0, storage);
+}
+
+module AP_MODULE_DECLARE_DATA proxy_store_score_module = {
+STANDARD20_MODULE_STUFF,
+NULL, /* create per-directory config structure */
+NULL, /* merge per-directory config structures */
+NULL, /* create per-server config structure */
+NULL, /* merge per-server config structures */
+NULL, /* command apr_table_t */
+ap_proxy_store_score_register_hook /* register hooks */
+};
+
Index: mod_proxy_store.h
===
ap_proxy_get_worker
Hi, Why does ap_proxy_get_worker() gives the best matched worker? - Shouldn't it give the exact match - Cheers Jean-Frederic
Re: AW: ap_proxy_get_worker
Plüm wrote: -Ursprüngliche Nachricht- Von: Jean-frederic Clere Gesendet: Mittwoch, 12. Juli 2006 14:21 An: dev@httpd.apache.org Betreff: ap_proxy_get_worker Hi, Why does ap_proxy_get_worker() gives the best matched worker? - Shouldn't it give the exact match - Normally you do not have an exact match between the request URL and the worker name (which is the last parameter of ProxyPass). So longest match is the best thing you can do here. Ok. What happends in a configuration like: +++ ProxyPass /foo http://foo.example.com/bar ProxyPass /bar http://foo.example.com/bar/foo +++ Only one worker will be created. Cheers Jean-Frederic ||| | Regards Rüdiger
Additing a storage for the shared information of the worker in mod_proxy
Hi,
I am still trying to replace the scoreboard by shared memory to store
the shared information of the workers, I am now thinking to get this by
adding modules like the prototype I have enclosed (that is a patch
against trunk).
Does this look to be the right way to go? Or has someone a better idea?
I am also thinking in using #defines so that the whole thing could be
optional (defaulting to actual code).
BTW: The mod_proxy_store_score.c is just to tell: The very first
mod_proxy_storage module I want to write will use... the scoreboard as
common area ;-)
Cheers
Jean-Frederic
Index: mod_proxy_store_score.c
===
--- mod_proxy_store_score.c (revision 0)
+++ mod_proxy_store_score.c (revision 0)
@@ -0,0 +1,58 @@
+/* Copyright 1999-2006 The Apache Software Foundation or its licensors, as
+ * applicable.
+ *
+ * Licensed under the Apache License, Version 2.0 (the License);
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* Comarea for the mod_proxy modules */
+#define CORE_PRIVATE
+
+#include mod_proxy.h
+#include mod_proxy_store.h
+// #include mod_core.h
+
+static void setstatus(proxy_worker *worker, int status)
+{
+}
+static int getstatus(proxy_worker *worker)
+{
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+ getstatus of the worker: %s, worker-name);
+return 0;
+}
+
+static const proxy_storage_method storage = {
+setstatus,
+getstatus
+};
+
+static void child_init(apr_pool_t *p, server_rec *s)
+{
+}
+
+static void ap_proxy_store_score_register_hook(apr_pool_t *p)
+{
+ap_hook_child_init(child_init, NULL, NULL, APR_HOOK_MIDDLE);
+ap_register_provider(p, PROXY_STORAGE, score, 0, storage);
+}
+
+module AP_MODULE_DECLARE_DATA proxy_store_score_module = {
+STANDARD20_MODULE_STUFF,
+NULL, /* create per-directory config structure */
+NULL, /* merge per-directory config structures */
+NULL, /* create per-server config structure */
+NULL, /* merge per-server config structures */
+NULL, /* command apr_table_t */
+ap_proxy_store_score_register_hook /* register hooks */
+};
+
Index: mod_proxy_store.h
===
--- mod_proxy_store.h (revision 0)
+++ mod_proxy_store.h (revision 0)
@@ -0,0 +1,6 @@
+#define PROXY_STORAGE proxystorage
+struct proxy_storage_method {
+void (*setstatus)(proxy_worker *worker, int status);
+int (*getstatus)(proxy_worker *worker);
+};
+typedef struct proxy_storage_method proxy_storage_method;
Index: config.m4
===
--- config.m4 (revision 421342)
+++ config.m4 (working copy)
@@ -19,6 +19,7 @@
proxy_fcgi_objs=mod_proxy_fcgi.lo
proxy_ajp_objs=mod_proxy_ajp.lo ajp_header.lo ajp_link.lo ajp_msg.lo
proxy_balancer_objs=mod_proxy_balancer.lo
+proxy_store_score=mod_proxy_store_score.lo
case $host in
*os2*)
@@ -39,6 +40,7 @@
APACHE_MODULE(proxy_fcgi, Apache proxy FastCGI module, $proxy_fcgi_objs, ,
$proxy_mods_enable)
APACHE_MODULE(proxy_ajp, Apache proxy AJP module, $proxy_ajp_objs, ,
$proxy_mods_enable)
APACHE_MODULE(proxy_balancer, Apache proxy BALANCER module,
$proxy_balancer_objs, , $proxy_mods_enable)
+APACHE_MODULE(proxy_store_score, Apache proxy score board storage module,
$proxy_store_score_objs, , $proxy_mods_enable)
APR_ADDTO(INCLUDES, [-I\$(top_srcdir)/$modpath_current/../generators])
Index: mod_proxy.c
===
--- mod_proxy.c (revision 421342)
+++ mod_proxy.c (working copy)
@@ -17,6 +17,7 @@
#define CORE_PRIVATE
#include mod_proxy.h
+#include mod_proxy_store.h
#include mod_core.h
#include apr_optional.h
#include scoreboard.h
@@ -38,6 +39,11 @@
#endif
/*
+ * Storage for the comarea
+ */
+static const proxy_storage_method *storage = NULL;
+
+/*
* A Web proxy module. Stages:
*
* translate_name: set filename to proxy:URL
@@ -1075,6 +1081,14 @@
const char *err = ap_proxy_add_worker(worker, cmd-pool, conf, r);
if (err)
return apr_pstrcat(cmd-temp_pool, ProxyPass , err, NULL);
+if (storage) {
+worker-context = (void *) storage;
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+add_pass: storage %d, storage);
+}
+else
+ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,
+
Re: Additing a storage for the shared information of the worker in mod_proxy
Brian Akins wrote: Jean-frederic Clere wrote: Hi, I am still trying to replace the scoreboard by shared memory to store the shared information of the workers, I am now thinking to get this by adding modules like the prototype I have enclosed (that is a patch against trunk). We really need a generic scoreboard type module with a relatively simple API to add and access per worker memory. I have some ideas if anyone wants to hear them... The httpd-dev list is here for that ;-) For the moment my need is just to have the worker in a place where I could monitor them from an outside httpd process (actively like pinging them or like in mod_backhand) and/or update their status so the loadbalancing improves its QoS. Cheers Jean-Frederic
Re: [Fwd: Re: proxy_worker_stat in mod_proxy]
Ruediger Pluem wrote: On 07/10/2006 12:21 PM, Jean-frederic Clere wrote: Ruediger Pluem wrote: context could be used. But I am thinking of sharing the balacing information with another program. By changing proxy_worker_stat * to a void * and arranging the corresponding code the first step to store the shared information out of the scoreboard would be done then what every common area could be used for the shared information: I want to use shared memory instead the scoreboard. This sounds reasonable to me as the stats should be available across process boundaries and relying on the scoreboard is simple but puts on a few constraints (especially when things should be extended). Furthermore I think that the scoreboard is sometimes in the danger of being used as the dumping ground for shared memory requirements of all kind. But I do not think that is had been designed for that task. OTH not using the scoreboard does not free us from the need of managing the storage of arbitrary balancer private data in the shared memory, because hardcoding does not seem to be an option for me. Once the shared data is accessed via a small API nothing prevent us going on using the scoreboard but will allow to use any other shared storage. be changed. So this shouldn't really be done. Maybe worth a warning in the error log? Easy. See enclosed patch. Thanks for the patch, but I don't think that we actually should change the behaviour as your patch does. This could break existing configs (how stupid they are). So I would prefer only logging a warning that this is a bad configuration. Agreed I will fix that and commit it. Cheers Jean-Frederic Regards Rüdiger
Re: [Fwd: Re: proxy_worker_stat in mod_proxy]
Ruediger Pluem wrote:
On 09.07.2006 16:37, Jim Jagielski wrote:
Ruediger Pluem wrote:
So it makes sense from my perspective to
add a void pointer *to* the proxy_worker_stat structure to enable
balancers to store custom data that is private to them.
This already exists. See
void*context; /* general purpose storage */
Ups, I missed that one. Thanks.
context could be used. But I am thinking of sharing the balacing
information with another program.
By changing proxy_worker_stat * to a void * and arranging the
corresponding code the first step to store the shared information out of
the scoreboard would be done then what every common area could be used
for the shared information: I want to use shared memory instead the
scoreboard.
Thus the only problem that remains with that is
that one worker might be shared by multiple balancers with different methods (if
this configuration makes sense is another story). This could cause problems if
the different balancers use different balancing methods. OTH from a quick look
at the code I assume that this configuration would be problematic anyway as
reusing an existing worker in a balancer causes the parameters of the worker to
be changed. So this shouldn't really be done. Maybe worth a warning in the
error log?
Easy. See enclosed patch.
Regards
Rüdiger
Index: modules/proxy/mod_proxy.c
===
--- modules/proxy/mod_proxy.c (revision 419906)
+++ modules/proxy/mod_proxy.c (working copy)
@@ -1069,8 +1069,11 @@
const char *err = ap_proxy_add_worker(worker, cmd-pool, conf, r);
if (err)
return apr_pstrcat(cmd-temp_pool, ProxyPass , err, NULL);
+PROXY_COPY_CONF_PARAMS(worker, conf);
+} else {
+ap_log_error(APLOG_MARK, APLOG_WARNING, 0, cmd-server,
+ worker %s already used by another worker (old conf
kept), worker-name);
}
-PROXY_COPY_CONF_PARAMS(worker, conf);
for (i = 0; i arr-nelts; i++) {
const char *err = set_worker_param(cmd-pool, worker, elts[i].key,
@@ -1462,8 +1465,11 @@
const char *err;
if ((err = ap_proxy_add_worker(worker, cmd-pool, conf, name)) !=
NULL)
return apr_pstrcat(cmd-temp_pool, BalancerMember , err, NULL);
+PROXY_COPY_CONF_PARAMS(worker, conf);
+} else {
+ap_log_error(APLOG_MARK, APLOG_WARNING, 0, cmd-server,
+ worker %s already used by another worker (old conf
kept), worker-name);
}
-PROXY_COPY_CONF_PARAMS(worker, conf);
arr = apr_table_elts(params);
elts = (const apr_table_entry_t *)arr-elts;
proxy_worker_stat in mod_proxy
Hi, I am looking in mod_proxy and I think that proxy_worker_stat of proxy_worker should be replaced by some void * to allow to write new balancers. The idea is each worker needs to point to the view the balancer of its, but only the balancer needs to know the structure it needs to make the balancing. Comments? Cheers Jean-Frederic
Re: mod_proxy_balancer/mod_proxy_ajp TODO
Henri Gomez wrote: The TomcatoMips indicator was just something to tell that it's not the raw CPU power which is important, but the estimated LOAD capacity of an instance. Accounting informations should included average time to execute a request, number of thread in use (AJP/HTTP), estimated free memory... Just to be sure that when a tomcat (for example), is near overload, the next requests will be routed to another less loaded tomcat. If you want such information I think Tomcat (or the backend server) has to provide it. Cheers Jean-Frederic 2006/6/22, Darryl Miles [EMAIL PROTECTED]: Henri Gomez wrote: Well you we always indicate some sort of CPU power for a remote (a sort of bogomips) and use this in computation. Why should the CPU power matter, what if the high power CPU is getting all the complex requests and the lower power CPU is ending up with simple request (static content). You are better implementing it in control packets over AJP that can advertise that hosts current willingness to take on new requests on a 32/64bit scale. Call this a flow control back pressure packet, a stateless beacon of information which may or may not be used by the client AJP. Then have a pluggable implementation at the server end of calculating that value and frequency for advertising it. All the apache LB has to do is work from that load calculation. All existing AJPs have to do is just ignore the packet. In the case of different horse power CPUs that factor is better fed into the server AJP algorithm by biasing the advertised willingness to take on new requests after a certain low level is reached. Only the server end of the AJP know the true request rate and how near that system is to breaking point. This scheme also works where apache may not see all the work the backend is doing, like if you have a different apache front end clusters linked to the same single backend cluster. Darryl
Re: Compiling a C++ module with g++ on Solaris
Phil Endecott wrote: Dear All, I'm the author of Anyterm (http://anyterm.org/) which uses an Apache module written in C++. This works OK on Linux. However, a user has attempted to compile it on Solaris and it fails at run time. Specifically, the first time that any C++ memory allocation stuff happens - it creates a std::string in the post_config hook function - it segfaults deep inside libstdc++. Have you try with sun studio? This user is using g++ 3.4.3 and the Sun linker on Solaris 10 x86. I'm curous to know if any readers have any experience compiling C++ Apache modules on this platform, and whether there are any gotchas to look out for. Is it possible that there is some libstdc++ initialisation that hasn't happened? I could imagine that this would require special support from the linker or the dlopen stuff, and that that behaves differently with Sun's libc and linker than on Linux. Any ideas? He has tried with 2.0.5? and 2.2.? with the same result. Cheers, Phil.
Re: [PATCH 1/6] scoreboard over-sized
Jim Jagielski wrote: Chris Darroch wrote: Personally, if I was writing a completely private module, I'd probably just patch the scoreboard.h definition of lb_score to include whatever extra stuff I needed, and recompile httpd. That way I'm insulated from any changes to the size of either the array or the proxy_worker_stat structure; if they change at some point in a future release and I don't happen to notice, I'm safe so long as I keep patching in my extra fields. I think that subtle care should be taken regarding anything that's added to the scoreboard. It's designed for a specific purpose, not just a general shared memory segment that modules should and can use for any and all purposes :) Ok, That is why I was asking :-) Cheers Jean-Frederic
modules/aaa/mod_authz_owner.c
Hi,
On ReliantUnix the following code can't be compiled:
+++
typedef struct {
} authz_owner_config_rec;
+++
Because the structure is empty.
Any problem to apply the following patch:
+++
Index: aaa/mod_authz_owner.c
===
--- aaa/mod_authz_owner.c (revision 370359)
+++ aaa/mod_authz_owner.c (working copy)
@@ -29,14 +29,13 @@
#include mod_auth.h /* for AUTHZ_GROUP_NOTE */
-typedef struct {
-} authz_owner_config_rec;
+typedef struct authz_owner_config_rec_struct *authz_owner_config_rec_ptr;
APR_DECLARE_OPTIONAL_FN(char*, authz_owner_get_file_group, (request_rec
*r));
static void *create_authz_owner_dir_config(apr_pool_t *p, char *d)
{
-authz_owner_config_rec *conf = apr_palloc(p, sizeof(*conf));
+authz_owner_config_rec_ptr conf = apr_palloc(p, sizeof(conf));
return conf;
}
+++
Cheers
Jean-Frederic
Re: 2.1.5 available for testing
William A. Rowe, Jr. wrote:
++1 To Joe's comments.
Jeff's fix is technically right, but scares the nibbles out
of me. If, for example, an exploit is able to inject the
T-E on top of the legit C-L, I really suspect we should not
trust the origin server at all.
For origin servers (as opposed to clients) make this choice
between ignore C-L, or fail, configurable?
My observation is that there are far more varied clients in
the world than servers, each with unique faults. But the
RFC 2616 is clear...
Messages MUST NOT include both a Content-Length header field and a
non-identity transfer-coding. If the message does include a non-
identity transfer-coding, the Content-Length MUST be ignored.
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected.
...and server authors can be expected to be less buggy than clients.
Permissive in what we accept, strict in what we send implies some
strictness in what we trust to pass on to the client.
So +.5 to Jeff's patch, and let's discuss if the proxy response should
then be trusted at all with T-E and C-L, in httpd-2.x where we support
keepalives.
Once the patch applied we lose the information that the request was incorrect.
That means we won't be able to choose in proxy between sending C-L (and dechunk)
and T-E.
[FYI +1 in httpd-1.3, that proxy does not use keepalives.]
Bill
Bill
At 06:40 PM 6/22/2005, Jeff Trawick wrote:
On 6/22/05, Paul Querna [EMAIL PROTECTED] wrote:
Joe Orton wrote:
On Wed, Jun 22, 2005 at 03:02:50PM -0500, William Rowe wrote:
Prior to either patch we totally mishandled such requests. So the
only question which remains is; which behavior do we prefer?
As the RFC states this is not acceptable, my gut says reject ANY
request with both C-L and T-E of non-identity.
I don't see any reason to reject anything, 2616 dictates precisely how
to handle requests which are malformed in this way. I do think the
check against identity is actually redundant, though; not least
because the 2616 errata remove all references to the word.
I think correct behaviour is to just follow the letter of Section 4.4,
point 3, as below:
If a message is received with both a
Transfer-Encoding header field and a Content-Length header field,
the latter MUST be ignored.
(and it's also going to be better to check for T-E before C-L since
99.99% of requests received are not going to have a T-E header so it'll
fall through slightly quicker)
+1 to the posted patch.
+1 here as well
What about proxy reading from origin server?
Origin server sends this to Apache acting as proxy:
HTTP/1.1 200 OK\r\n
Content-Length: 99\r\n
Transfer-Encoding: chunked\r\n
\r\n
0A\r\n
aa\r\n
00\r\n
\r\n
Client receives this:
HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:12:31 GMT
Content-Length: 99
Content-Type: text/plain; charset=ISO-8859-1
aa(END)
Add this patch:
Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c (revision 191655)
+++ modules/proxy/mod_proxy_http.c (working copy)
@@ -1128,7 +1128,17 @@
r-headers_out,
save_table);
}
-
+
+/* can't have both Content-Length and Transfer-Encoding */
+if (apr_table_get(r-headers_out, Transfer-Encoding)
+ apr_table_get(r-headers_out, Content-Length)) {
+/* 2616 section 4.4, point 3: if both Transfer-Encoding
+ * and Content-Length are received, the latter MUST be
+ * ignored; so unset it here to prevent any confusion
+ * later. */
+apr_table_unset(r-headers_out, Content-Length);
+}
+
/* strip connection listed hop-by-hop headers from response */
backend-close +=
ap_proxy_liststr(apr_table_get(r-headers_out,
Connection),
Client now gets this:
HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:22:19 GMT
Transfer-Encoding: chunked
Content-Type: text/plain; charset=ISO-8859-1
a
aa
0
Re: 2.1.5 available for testing
Jeff Trawick wrote:
On 6/23/05, jean-frederic clere [EMAIL PROTECTED] wrote:
William A. Rowe, Jr. wrote:
++1 To Joe's comments.
Jeff's fix is technically right, but scares the nibbles out
of me. If, for example, an exploit is able to inject the
T-E on top of the legit C-L, I really suspect we should not
trust the origin server at all.
One important situation to fear is a buggy server or proxy+server that
we may not be able to talk to at all if we are extremely strict.
[As you implied w.r.t. Apache 1.3] The smuggling fear is really if we
allow keepalive on this connection to the origin server again. We
could be confused by making the wrong choice from {CL, TE} and
misunderstand the next response. We can set backend-close and
origin-keepalive to prevent that.
If we don't allow keepalive, then it is down to whether or not this
single request can be parsed correctly if our choice of {CL, TE} makes
sense.
For origin servers (as opposed to clients) make this choice
between ignore C-L, or fail, configurable?
try very hard to make a reasonable choice with no configuration :)
(speaking to the choir, of course)
My observation is that there are far more varied clients in
the world than servers, each with unique faults. But the
RFC 2616 is clear...
Messages MUST NOT include both a Content-Length header field and a
non-identity transfer-coding. If the message does include a non-
identity transfer-coding, the Content-Length MUST be ignored.
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected.
...and server authors can be expected to be less buggy than clients.
Permissive in what we accept, strict in what we send implies some
strictness in what we trust to pass on to the client.
We're removing the protocol breakage in what we pass on to the client.
At this point, we either send a valid response or it is if the server
dropped the connection before sending the full response.
(I hear what you're screaming. I think the minimal-intervention path
should be preferred if we can justify it.)
So +.5 to Jeff's patch, and let's discuss if the proxy response should
then be trusted at all with T-E and C-L, in httpd-2.x where we support
keepalives.
Once the patch applied we lose the information that the request was incorrect.
That means we won't be able to choose in proxy between sending C-L (and dechunk)
and T-E.
I don't follow here. How does the backend choice of {TE, CL} affect
what we send the client?
If we are acting as a proxy, what we send to the next proxy is changed by the
patch, isn't it?
Re: 2.1.5 available for testing
William A. Rowe, Jr. wrote: At 03:07 PM 6/17/2005, William A. Rowe, Jr. wrote: -1 on Win32, caddr_t isn't sufficiently portable (fix committed). Correction, -1 on all platforms. jfclere just committed a significant patch to the T-E override of the C-L passed to us, as part of the Watchfire vulnerability fixes. It seems very irresponsible to release any flavor (alpha, beta, nadda) with a known vuln, when we've committed a fix already. I still need some more time to check POST with 2 different content-lengths and GET with content-length. Also, possibly across platforms is a fault in ssl_engine_init, where the host-protocol is still NULL, and we are trying to strcmp it to 'https'. I spent part of my weekend trying to grok what change has broken this, but strcmp to NULL is popping a segfault. Not worthy of rejecting 2.1.5 on it's own, this is still a minor irritation. FYI - mod_ssl was loaded without SSL being defined, so no ssl host actually exists. Bill
Re: 2.1.5 available for testing
jean-frederic clere wrote: William A. Rowe, Jr. wrote: At 03:07 PM 6/17/2005, William A. Rowe, Jr. wrote: -1 on Win32, caddr_t isn't sufficiently portable (fix committed). Correction, -1 on all platforms. jfclere just committed a significant patch to the T-E override of the C-L passed to us, as part of the Watchfire vulnerability fixes. It seems very irresponsible to release any flavor (alpha, beta, nadda) with a known vuln, when we've committed a fix already. I still need some more time to check POST with 2 different content-lengths HTTP_BAD_REQUEST for this one. and GET with content-length. I think that is not forbidden in the rfc... But what about returning HTTP_BAD_REQUEST if Content-Length is not 0? Also, possibly across platforms is a fault in ssl_engine_init, where the host-protocol is still NULL, and we are trying to strcmp it to 'https'. I spent part of my weekend trying to grok what change has broken this, but strcmp to NULL is popping a segfault. Not worthy of rejecting 2.1.5 on it's own, this is still a minor irritation. FYI - mod_ssl was loaded without SSL being defined, so no ssl host actually exists. Bill
Re: 2.1.5 available for testing
William A. Rowe, Jr. wrote: At 08:55 AM 6/20/2005, jean-frederic clere wrote: jean-frederic clere wrote: and GET with content-length. I think that is not forbidden in the rfc... But what about returning HTTP_BAD_REQUEST if Content-Length is not 0? See section 4.3 of RFC 2616. Content-Length: 0 signals a message body of 0 bytes, not the absence of a message body. It shouldn't be treated differently from Content-Length: n. OK, HTTP_BAD_REQUEST if C-L is present in GET? Bill
[error] (88)Socket operation on non-socket:
Hi, On my old Suse (7.2 (i386)) httpd (2.1 head) is not running and in the error_log I have the following: +++ [Fri Jun 17 13:58:50 2005] [error] (88)Socket operation on non-socket: apr_socket_accept: (client socket 0) +++ This only appends when starting http via crontab. Does it make sense to open fd 0, 1 and 2 in httpd to prevent such problems (in apr_proc_detach() for example)? Cheers Jean-Frederic
Re: svn commit: r124080 - /httpd/httpd/trunk/modules/cache/mod_disk_cache.c
Jeff Trawick wrote: On 4 Jan 2005 09:58:09 -, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Author: jfclere Date: Tue Jan 4 01:58:01 2005 New Revision: 124080 URL: http://svn.apache.org/viewcvs?view=revrev=124080 Log: Add including of util_charset.h otherwise ap_hdrs_from_ascii is not defined. Modified: httpd/httpd/trunk/modules/cache/mod_disk_cache.c Modified: httpd/httpd/trunk/modules/cache/mod_disk_cache.c Url: http://svn.apache.org/viewcvs/httpd/httpd/trunk/modules/cache/mod_disk_cache.c?view=diffrev=124080p1=httpd/httpd/trunk/modules/cache/mod_disk_cache.cr1=124079p2=httpd/httpd/trunk/modules/cache/mod_disk_cache.cr2=124080 == --- httpd/httpd/trunk/modules/cache/mod_disk_cache.c(original) +++ httpd/httpd/trunk/modules/cache/mod_disk_cache.cTue Jan 4 01:58:01 2005 @@ -20,6 +20,10 @@ #include util_filter.h #include util_script.h +#if APR_CHARSET_EBCDIC +#include util_charset.h +#endif why not include it unconditionally? Just my bad. util_charset.h has already the needed #if APR_CHARSET_EBCDIC. Cheers Jean-Frederic
mod_deflate.c problems
Hi, I have the following problem with mod_deflate: +++ In file included from /usr/include/zutil.h:16, from mod_deflate.c:50: /usr/include/zlib.h:40: warning: `ZLIB_VERSION' redefined /opt/SMAWPlus/include/zlib.h:40: warning: this is the location of the previous definition In file included from /usr/include/zlib.h:34, from /usr/include/zutil.h:16, from mod_deflate.c:50: +++ Does it make sense to add a test to make sure zutil.h will not conflict with the libz we want to use to build mod_deflate.c? Cheers Jean-Frederic
Re: PATCH to use apr-iconv
William A. Rowe, Jr. wrote: At 06:38 PM 10/4/2004, Justin Erenkrantz wrote: I have some issues with the proposed patch in that it moves some configure logic that really belongs in apr-util over to httpd: i.e. configuration of apr-iconv should be done by apr-util not by httpd, httpd should only be aware of iconv via the apu-*-config scripts, etc. -- justin 100% agreed - if you look at the doxygen, these interfaces are private to apr-util, users aren't to count on apr-iconv itself. I have now moved all the logic to apr-util. Find enclosed the new patch. To get it working checkout apr-iconv in srclib and add --with-iconv=`pwd`/srclib/apr-iconv. More comments? Bill Index: srclib/apr-util/Makefile.in === RCS file: /home/cvspublic/apr-util/Makefile.in,v retrieving revision 1.97 diff -u -r1.97 Makefile.in --- srclib/apr-util/Makefile.in 23 Aug 2004 20:22:18 - 1.97 +++ srclib/apr-util/Makefile.in 5 Oct 2004 08:40:12 - @@ -15,8 +15,8 @@ APRUTIL_LIBS = @APRUTIL_LIBS@ TARGET_LIB = [EMAIL PROTECTED]@.la -INSTALL_SUBDIRS = @APR_XML_DIR@ -EXTRA_SOURCE_DIRS = @APR_XML_DIR@ +INSTALL_SUBDIRS = @APR_ICONV_DIR@ @APR_XML_DIR@ +EXTRA_SOURCE_DIRS = @APR_ICONV_DIR@ @APR_XML_DIR@ [EMAIL PROTECTED]@ APRUTIL_PCFILE = apr-util-$(APRUTIL_MAJOR_VERSION).pc APU_CONFIG = apu-$(APRUTIL_MAJOR_VERSION)-config Index: srclib/apr-util/buildconf === RCS file: /home/cvspublic/apr-util/buildconf,v retrieving revision 1.13 diff -u -r1.13 buildconf --- srclib/apr-util/buildconf 1 Jul 2004 22:45:06 - 1.13 +++ srclib/apr-util/buildconf 5 Oct 2004 08:40:12 - @@ -1,4 +1,20 @@ #!/bin/sh +# +# Copyright 1999-2004 The Apache Software Foundation +# +# Licensed under the Apache License, Version 2.0 (the License); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an AS IS BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# # Default place to look for apr source. Can be overridden with # --with-apr=[directory] @@ -63,6 +79,14 @@ # echo Generating 'make' outputs ... $apr_src_dir/build/gen-build.py make + +# +# If apr-iconv, then go and configure it. +# +if test -d ../apr-iconv; then + echo Invoking ../apr-iconv/buildconf.sh ... + (cd ../apr-iconv; ./buildconf) +fi # # If Expat has been bundled, then go and configure the thing Index: srclib/apr-util/configure.in === RCS file: /home/cvspublic/apr-util/configure.in,v retrieving revision 1.79 diff -u -r1.79 configure.in --- srclib/apr-util/configure.in2 Sep 2004 13:50:28 - 1.79 +++ srclib/apr-util/configure.in5 Oct 2004 08:40:12 - @@ -96,6 +96,21 @@ APR_ADDTO(CFLAGS, `$apr_config --cflags`) APR_ADDTO(CPPFLAGS, `$apr_config --cppflags`) +dnl +dnl Find the APR-ICONV directory. +dnl +if test -d ../apr-iconv; then + APR_SUBDIR_CONFIG(../apr-iconv, +[$apache_apr_flags --prefix=$prefix --exec-prefix=$exec_prefix --libdir=$libdir --includedir=$includedir --bindir=$bindir --datadir=$datadir --with-installbuilddir=$installbuilddir], +[--enable-layout=*|\'--enable-layout=*]) + APRUTIL_EXPORT_LIBS=$abs_srcdir/../apr-iconv/lib/libapriconv.la $APRUTIL_EXPORT_LIBS + APRUTIL_INCLUDES=-I$abs_srcdir/../apr-iconv/include $APRUTIL_INCLUDES + APR_ICONV_DIR=../apr-iconv +else + APR_ICONV_DIR= +fi +AC_SUBST(APR_ICONV_DIR) + dnl Find LDAP library dnl Determine what DBM backend type to use. dnl Find Expat Index: srclib/apr-util/build/apu-iconv.m4 === RCS file: /home/cvspublic/apr-util/build/apu-iconv.m4,v retrieving revision 1.5 diff -u -r1.5 apu-iconv.m4 --- srclib/apr-util/build/apu-iconv.m4 21 Jul 2002 11:09:12 - 1.5 +++ srclib/apr-util/build/apu-iconv.m4 5 Oct 2004 08:40:12 - @@ -22,33 +22,44 @@ AC_DEFUN(APU_FIND_ICONV,[ apu_iconv_dir=unknown +have_apr_iconv=0 AC_ARG_WITH(iconv,[ --with-iconv[=DIR]path to iconv installation], [ apu_iconv_dir=$withval if test $apu_iconv_dir != yes; then APR_ADDTO(CPPFLAGS,[-I$apu_iconv_dir/include]) APR_ADDTO(LDFLAGS,[-L$apu_iconv_dir/lib]) fi +if test -f $apu_iconv_dir/include/api_version.h; then + have_apr_iconv=1 + have_iconv=0 + APR_REMOVEFROM(LIBS,[-lapriconv]) + AC_MSG_RESULT(Using apr-iconv) +fi ]) -AC_CHECK_HEADER(iconv.h, [ - APU_TRY_ICONV([ have_iconv=1 ], [ - - APR_ADDTO(LIBS,[-liconv]) - - APU_TRY_ICONV([ -
Re: cvs commit: httpd-2.0/os/unix os.h unixd.c
Jeff Trawick wrote:
On 1 Oct 2004 16:03:09 -, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
jfclere 2004/10/01 09:03:09
Modified:os config.m4
os/unix os.h unixd.c
Log:
Move the few BS2000 specific in unixd.c
Index: unixd.c
===
RCS file: /home/cvs/httpd-2.0/os/unix/unixd.c,v
retrieving revision 1.69
retrieving revision 1.70
diff -u -r1.69 -r1.70
--- unixd.c 24 Apr 2004 19:42:52 - 1.69
+++ unixd.c 1 Oct 2004 16:03:08 - 1.70
@@ -457,11 +457,26 @@
{
apr_socket_t *csd;
apr_status_t status;
+#ifdef _OSD_POSIX
+int sockdes;
+#endif
*accepted = NULL;
status = apr_socket_accept(csd, lr-sd, ptrans);
if (status == APR_SUCCESS) {
*accepted = csd;
+#ifdef _OSD_POSIX
+apr_os_sock_get(sockdes, csd);
+if (sockdes = FD_SETSIZE) {
+ap_log_error(APLOG_MARK, APLOG_WARNING, 0, NULL,
+ new file descriptor %d is too large; you probably need
+ to rebuild Apache with a larger FD_SETSIZE
+ (currently %d),
+ sockdes, FD_SETSIZE);
+apr_socket_close(csd);
+return APR_EINTR;
+}
+#endif
the old logic was removed on purpose
an analogous check is in APR at the point where select() is actually
needed; an error will be returned from the apr poll call or the apr
send/recv call
the APR check covers not just sockets but also pipes used to
communicate with CGIs
it would be nice if there was a special APR error code to use there
with a unique error message though
(btw, it is APR that would need to be rebuilt)
In poll/unix/poll.c:
+++
if (fd = FD_SETSIZE) {
/* XXX invent new error code so application has a clue */
return APR_EBADF;
}
+++
PATCH to use apr-iconv
Hi,
I have prepared a patch to use apr-iconv instead GNU or system iconv.
Find it enclosed.
Any comments?
Cheers
Jean-Frederic
Index: buildconf
===
RCS file: /home/cvspublic/httpd-2.0/buildconf,v
retrieving revision 1.44
diff -u -r1.44 buildconf
--- buildconf 3 Sep 2004 01:04:36 - 1.44
+++ buildconf 4 Oct 2004 16:50:34 -
@@ -99,6 +99,7 @@
apr_configure=$apr_src_dir/configure
aprutil_configure=$apu_src_dir/configure
pcre_configure=srclib/pcre/configure
+apriconv_configure=srclib/apr-iconv/configure
config_h_in=include/ap_config_auto.h.in
cross_compile_warning=warning: AC_TRY_RUN called without default to allow cross
compiling
@@ -115,6 +116,15 @@
echo rebuilding $aprutil_configure
(cd srclib/apr-util ./buildconf) || {
echo ./buildconf failed for apr-util
+exit 1
+}
+fi
+
+
+if [ -d srclib/apr-iconv ]; then
+echo rebuilding $apriconv_configure
+(cd srclib/apr-iconv ./buildconf) || {
+echo ./buildconf failed for apr-iconv
exit 1
}
fi
Index: configure.in
===
RCS file: /home/cvspublic/httpd-2.0/configure.in,v
retrieving revision 1.270
diff -u -r1.270 configure.in
--- configure.in21 Sep 2004 20:04:51 - 1.270
+++ configure.in4 Oct 2004 16:50:35 -
@@ -110,6 +110,17 @@
APU_VERSION=`$apu_config --version`
APU_CONFIG=$APU_BINDIR/apu-`echo ${APU_VERSION} | sed 's,\..*,,'`-config
+if test -d srclib/apr-iconv; then
+ APR_SUBDIR_CONFIG(srclib/apr-iconv,
+[--with-apr=../apr --prefix=$prefix --exec-prefix=$exec_prefix
--libdir=$libdir --includedir=$includedir --bindir=$bindir],
+[--enable-layout=*|\'--enable-layout=*])
+ dnl We must be the last to build and the first to be cleaned
+ AP_BUILD_SRCLIB_DIRS=$AP_BUILD_SRCLIB_DIRS apr-iconv
+ AP_CLEAN_SRCLIB_DIRS=apr-iconv $AP_CLEAN_SRCLIB_DIRS
+ dnl dirty hack
+ AP_LIBS=`pwd`/srclib/apr-iconv/lib/libapriconv.la $AP_LIBS
+fi
+
dnl In case we picked up CC and CPP from APR, get that info into the
dnl config cache so that PCRE uses it. Otherwise, CC and CPP used for
dnl PCRE and for our config tests will be whatever PCRE determines.
Index: srclib/apr-util/build/apu-iconv.m4
===
RCS file: /home/cvspublic/apr-util/build/apu-iconv.m4,v
retrieving revision 1.5
diff -u -r1.5 apu-iconv.m4
--- srclib/apr-util/build/apu-iconv.m4 21 Jul 2002 11:09:12 - 1.5
+++ srclib/apr-util/build/apu-iconv.m4 4 Oct 2004 16:50:36 -
@@ -28,27 +28,38 @@
APR_ADDTO(CPPFLAGS,[-I$apu_iconv_dir/include])
APR_ADDTO(LDFLAGS,[-L$apu_iconv_dir/lib])
fi
+have_apr_iconv=0
+if test -f $apu_iconv_dir/include/api_version.h; then
+ have_apr_iconv=1
+ have_iconv=0
+ APR_REMOVEFROM(LIBS,[-lapriconv])
+ AC_MSG_RESULT(Using apr-iconv)
+fi
])
-AC_CHECK_HEADER(iconv.h, [
- APU_TRY_ICONV([ have_iconv=1 ], [
+if test $have_apr_iconv != 1; then
+ AC_CHECK_HEADER(iconv.h, [
+APU_TRY_ICONV([ have_iconv=1 ], [
+
+APR_ADDTO(LIBS,[-liconv])
+
+APU_TRY_ICONV([
+ APR_ADDTO(APRUTIL_LIBS,[-liconv])
+ APR_ADDTO(APRUTIL_EXPORT_LIBS,[-liconv])
+ have_iconv=1 ],
+ [ have_iconv=0 ])
- APR_ADDTO(LIBS,[-liconv])
+APR_REMOVEFROM(LIBS,[-liconv])
- APU_TRY_ICONV([
- APR_ADDTO(APRUTIL_LIBS,[-liconv])
- APR_ADDTO(APRUTIL_EXPORT_LIBS,[-liconv])
- have_iconv=1 ],
- [ have_iconv=0 ])
-
- APR_REMOVEFROM(LIBS,[-liconv])
-
- ])
-], [ have_iconv=0 ])
+])
+ ], [ have_iconv=0 ])
+fi
if test $apu_iconv_dir != unknown; then
if test $have_iconv != 1; then
-AC_MSG_ERROR([iconv support requested, but not found])
+if test $have_apr_iconv != 1; then
+ AC_MSG_ERROR([iconv support requested, but not found])
+fi
fi
APR_REMOVEFROM(CPPFLAGS,[-I$apu_iconv_dir/include])
APR_REMOVEFROM(LDFLAGS,[-L$apu_iconv_dir/lib])
@@ -65,6 +76,7 @@
APR_CHECK_DEFINE(CODESET, langinfo.h, [CODESET defined in langinfo.h])
AC_SUBST(have_iconv)
+AC_SUBST(have_apr_iconv)
])dnl
dnl
Index: srclib/apr-util/include/apu.h.in
===
RCS file: /home/cvspublic/apr-util/include/apu.h.in,v
retrieving revision 1.20
diff -u -r1.20 apu.h.in
--- srclib/apr-util/include/apu.h.in26 Feb 2004 21:55:12 - 1.20
+++ srclib/apr-util/include/apu.h.in4 Oct 2004 16:50:36 -
@@ -77,7 +77,7 @@
#define APU_HAVE_DB_VERSION@apu_db_version@
#endif /* APU_HAVE_DB */
-#define APU_HAVE_APR_ICONV 0
+#define APU_HAVE_APR_ICONV @have_apr_iconv@
#define APU_HAVE_ICONV @have_iconv@
#define APR_HAS_XLATE (APU_HAVE_APR_ICONV || APU_HAVE_ICONV)
Re: [PATCH] don't crash with per-dir (location) rewrite config and NULL r-filename
Jeff Trawick wrote:
See attached patch. Given a module with map-to-storage hook which
leaves r-filename NULL, and config like the following, you get
segfault on platforms that don't like strlen(NULL).
Location /silly/foo
RewriteEngine On
RewriteCond %{SERVER_PORT} ^8080$
RewriteRule (.*) https://%{SERVER_NAME}%{REQUEST_URI}
/Location
/silly is handled by a module which implements a map-to-storage hook
and leaves r-filename NULL
Why not returning DECLINED when r-filename is NULL?
comments? better way to do it?
odd to me that rewrite's translate_name hook, used for processing
server config directives, can update r-filename permanently even when
it declines
Re: proxy compile warnings
Geoffrey Young wrote:
William A. Rowe, Jr. wrote:
At 04:36 PM 8/17/2004, Joe Orton wrote:
On Tue, Aug 17, 2004 at 11:48:39AM -0400, Geoffrey Young wrote:
hi all
the attached patch is required for me to get mod_proxy to compile with -Werror.
Thanks Geoff. The only thing I'm not sure of is whether this function
should really be deleted or if it's actually intended to be exported:
-PROXY_DECLARE(int) ap_proxy_post_request(proxy_worker *worker,
- proxy_balancer *balancer,
- request_rec *r,
- proxy_server_conf *conf)
-{
I have the same concern... Madhu is gone for about another week,
we should have an answer, soon.
sure, that's fine. I didn't consider the alternative, which is adding to
mod_proxy.h. whoops :)
Adding it to mod_proxy.h is right the thing.
The idea is to have a ap_proxy_pre_request and ap_proxy_post_request: one called
before processing the request and one after.
You can check cvs revision log
for the include file to see if he deliberate dropped it.
this is the only reference I can find to that function at all
http://www.mail-archive.com/[EMAIL PROTECTED]/msg61185.html
but I haven't been following this complex proxy world with a lot of detail.
however, it is a recent addition, so I would guess that the include is just
missing the declaration.
at any rate, no sense rushing it along in lieu of the proper people
reviewing it - I was just trying to let people know about the warnings :)
--Geoff
Re: Where is the AJP code...
William A. Rowe, Jr. wrote: At 01:56 AM 8/17/2004, jean-frederic clere wrote: William A. Rowe, Jr. wrote: At 03:58 PM 8/16/2004, Brad Nicholes wrote: The addition of the proxy_ajp module to the httpd project presents a build problem. The AJP module code exists in httpd/modules/proxy in the httpd project but the AJP lib code still sits in the jakarta-tomcat-connectors\ajp\ajplib\test in the jakarta project. Is the AJPLIB code going to be migrated to the httpd project? Why is the AJPLIB code in a test directory? I'm +1 for placing it in modules/proxy/ ... iff it will be maintained and kept up to date. +1 Then let's go ahead, would you please commit the current flavor of the ajp sources to modules/proxy/ (not modules/proxy/ajp/) with a commit message to their official source location within jakarta-tomcat-connectors/ (the real location, not the temporary ajp/test/ajplib). The real location has not been decided in jakarta-tomcat-connectors... For the moment the ajplib files are in jakarta-tomcat-connectors/ajp/test/ajplib. Just in case another maintainer wants to look aside at their development. Then just a few tweaks so we don't delve the ./ajp/ directory. Bill
AJP lib files added to httpd-2.0 repos
Hi, I have added to the httpd-2.0/modules/proxy the following files: ajp.h ajp_header.c ajp_header.h ajp_link.c ajp_msg.c They are also in jakarta-tomcat-connectors/ajp/ajplib/test. They have to be kept up to date (synchronized) please remember to mail the diff to the other repos dev-list when something is changed. Cheers Jean-Frederic
vacation
Hi, I'll be on vacation til 7th september. I will (try to) go on the new proxy implementation but I will only read my apache.org mail, so if you find something wrong in the new code (and want a quick fix) mail me there. Cheers Jean-Frederic
Re: Where is the AJP code...
William A. Rowe, Jr. wrote: At 03:58 PM 8/16/2004, Brad Nicholes wrote: The addition of the proxy_ajp module to the httpd project presents a build problem. The AJP module code exists in httpd/modules/proxy in the httpd project but the AJP lib code still sits in the jakarta-tomcat-connectors\ajp\ajplib\test in the jakarta project. Is the AJPLIB code going to be migrated to the httpd project? Why is the AJPLIB code in a test directory? I'm +1 for placing it in modules/proxy/ ... iff it will be maintained and kept up to date. +1 If not, assuming that lib location is wrong - it should become an autoconf option. For the moment that would bring chicken/egg problems because the library needs a configured httpd-2.x source tree to build. Can we trust that ajp lib c sources will be kept in sync? I will try my best to synchronize both code locations. Bill
Re: proxy CONNECT and recent 2.1-dev proxy enhancements
Jeff Trawick wrote: probably obvious to those doing the load balancing work, but just in case... url in this case is ip:port, which ap_proxy_get_balancer() doesn't handle simple working testcase in case this helps testing connect and send CONNECT 0.0.0.0:8080\r\n Do we need balancer for CONNECT? get back: : 48545450 2F312E30 20323030 20436F6E HTTP/1.0 200 Con 0010: 6E656374 696F6E20 45737461 626C6973 nection Establis 0020: 6865640D 0A50726F 78792D61 67656E74 hed..Proxy-agent 0030: 3A204170 61636865 2F322E30 2E35302D : Apache/2.0.50- 0040: 64657620 28556E69 78292044 41562F32 dev (Unix) DAV/2 0050: 0D0A0D0A send GET / HTTP/1.1\r\nHost: foo\r\n\r\n get back normal response (here is start): : 48545450 2F312E31 20323030 204F4B0D HTTP/1.1 200 OK. 0010: 0A446174 653A2053 61742C20 31342041 .Date: Sat, 14 A 0020: 75672032 30303420 31353A32 333A3033 ug 2004 15:23:03 0030: 20474D54 0D0A5365 72766572 3A204170 GMT..Server: Ap 0040: 61636865 2F322E30 2E35302D 64657620 ache/2.0.50-dev 0050: 28556E69 78292044 41562F32 0D0A4C61 (Unix) DAV/2..La 0060: 73742D4D 6F646966 6965643A 20467269 st-Modified: Fri 0070: 2C203034 204D6179 20323030 31203030 , 04 May 2001 00 0080: 3A30313A 31382047 4D540D0A 45546167 :01:18 GMT..ETag 0090: 3A202263 6336622D 3562302D 34303434 : cc6b-5b0-4044 needs AllowConnect directive specifying whatever port you wish to CONNECT to
Re: proxy CONNECT and recent 2.1-dev proxy enhancements
Jeff Trawick wrote:
On Mon, 16 Aug 2004 12:16:02 +0200, jean-frederic clere
[EMAIL PROTECTED] wrote:
Jeff Trawick wrote:
probably obvious to those doing the load balancing work, but just in case...
url in this case is ip:port, which ap_proxy_get_balancer() doesn't handle
simple working testcase in case this helps testing
connect and send CONNECT 0.0.0.0:8080\r\n
Do we need balancer for CONNECT?
I would think not. My understanding is that CONNECT is just a pipe,
with no ability to keep connections open (e.g., in keepalive state) or
re-use connections already opened.
Ok, the patch is quite easy in this case.
BTW: proxy_connect.c loops when the client closes the connection, I am preparing
a patch for this problem.
Index: mod_proxy.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/proxy/mod_proxy.c,v
retrieving revision 1.130
diff -u -r1.130 mod_proxy.c
--- mod_proxy.c 13 Aug 2004 12:09:13 - 1.130
+++ mod_proxy.c 16 Aug 2004 11:13:35 -
@@ -611,8 +611,14 @@
/* Try to obtain the most suitable worker */
access_status = ap_proxy_pre_request(worker, balancer, r, conf, url);
-if (access_status != OK)
+if (access_status != OK r-method_number != M_CONNECT) {
+#if DEBUGGING
+ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
+ NoWorker for %s (%s) %d,
+ r-uri, url, access_status);
+#endif
return access_status;
+}
/* firstly, try a proxy, unless a NoProxy directive is active */
if (!direct_connect) {
Re: [proxy] New implementation ready for testing
Pier Fumagalli wrote:
On 11 Aug 2004, at 17:14, Mladen Turk wrote:
Hi all,
We've finished the initial development of extended mod_proxy.
Since the development took place at jakata-tomcat-connectors,
the source code can be found under ajp/proxy.
Here is the list of major features added:
3. Added new module proxy_balancer
What's wrong with:
- http is slower than ajp (Therefore the idea of mod_proxy_ajp).
- failover.
Cheers
Jean-Frederic
ProxyPassReverse / http://localhost:/
ProxyPreserveHost On
RewriteMap hosts rnd:/opt/apache/conf/tables/hosts.map
RewriteRule ^/(.*) ${hosts:live}/$1 [P,L]
It's in there already, and it works great (I even have a couple of CGIs
reading and building up that table, enabling, disabling and prioritizing
hosts).
http://wiki.apache.org/cocoon/ApacheModProxy
http://wiki.apache.org/cocoon/LoadBalancingWithModProxy
Pier
[PATCH] proxy_connect.c
Hi,
While arranging the code of the recent 2.1-dev proxy enhancements to get CONNECT
working I have noted that CONNECT loops endless when the client closes the
connection. Find enclosed a patch than fixes the problem.
Cheers
Jean-Frederic
Index: proxy_connect.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/proxy/proxy_connect.c,v
retrieving revision 1.68
diff -u -r1.68 proxy_connect.c
--- proxy_connect.c 13 Aug 2004 23:16:50 - 1.68
+++ proxy_connect.c 16 Aug 2004 12:05:38 -
@@ -292,8 +292,10 @@
ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, proxy: CONNECT: error
apr_poll());
return HTTP_INTERNAL_SERVER_ERROR;
}
-/* ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
- proxy: CONNECT: woke from select(), i=%d, pollcnt);*/
+#ifdef DEBUGGING
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: CONNECT: woke from select(), i=%d, pollcnt);
+#endif
for (i = 0; i pollcnt; i++) {
const apr_pollfd_t *cur = signalled[i];
@@ -301,10 +303,13 @@
if (cur-desc.s == sock) {
pollevent = cur-rtnevents;
if (pollevent APR_POLLIN) {
-/* ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
- proxy: CONNECT: sock was set);*/
+#ifdef DEBUGGING
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: CONNECT: sock was set);
+#endif
nbytes = sizeof(buffer);
-if (apr_socket_recv(sock, buffer, nbytes) == APR_SUCCESS) {
+rv = apr_socket_recv(sock, buffer, nbytes);
+if (rv == APR_SUCCESS) {
o = 0;
i = nbytes;
while(i 0)
@@ -316,7 +321,8 @@
* if ((nbytes = ap_rwrite(buffer + o, nbytes, r)) 0)
* rbb
*/
-if (apr_socket_send(client_socket, buffer + o, nbytes)
!= APR_SUCCESS)
+rv = apr_socket_send(client_socket, buffer + o, nbytes);
+if (rv != APR_SUCCESS)
break;
o += nbytes;
i -= nbytes;
@@ -331,16 +337,24 @@
else if (cur-desc.s == client_socket) {
pollevent = cur-rtnevents;
if (pollevent APR_POLLIN) {
-/* ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
- proxy: CONNECT: client was set);*/
+#ifdef DEBUGGING
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: CONNECT: client was set);
+#endif
nbytes = sizeof(buffer);
-if (apr_socket_recv(client_socket, buffer, nbytes) ==
APR_SUCCESS) {
+rv = apr_socket_recv(client_socket, buffer, nbytes);
+if (rv == APR_SUCCESS) {
o = 0;
i = nbytes;
+#ifdef DEBUGGING
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: CONNECT: read %d from client, i);
+#endif
while(i 0)
{
nbytes = i;
-if (apr_socket_send(sock, buffer + o, nbytes) !=
APR_SUCCESS)
+rv = apr_socket_send(sock, buffer + o, nbytes);
+if (rv != APR_SUCCESS)
break;
o += nbytes;
i -= nbytes;
@@ -354,6 +368,9 @@
}
else
break;
+}
+if (rv != APR_SUCCESS) {
+break;
}
}
Re: proxy CONNECT and recent 2.1-dev proxy enhancements
Jeff Trawick wrote:
On Mon, 16 Aug 2004 13:20:41 +0200, jean-frederic clere
[EMAIL PROTECTED] wrote:
Jeff Trawick wrote:
On Mon, 16 Aug 2004 12:16:02 +0200, jean-frederic clere
[EMAIL PROTECTED] wrote:
Jeff Trawick wrote:
probably obvious to those doing the load balancing work, but just in case...
url in this case is ip:port, which ap_proxy_get_balancer() doesn't handle
simple working testcase in case this helps testing
connect and send CONNECT 0.0.0.0:8080\r\n
Do we need balancer for CONNECT?
I would think not. My understanding is that CONNECT is just a pipe,
with no ability to keep connections open (e.g., in keepalive state) or
re-use connections already opened.
Ok, the patch is quite easy in this case.
BTW: proxy_connect.c loops when the client closes the connection, I am preparing
a patch for this problem.
good; I had a patch in my tree and went to test it this weekend but
didn't get past the balancer ;)
Index: mod_proxy.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/proxy/mod_proxy.c,v
retrieving revision 1.130
diff -u -r1.130 mod_proxy.c
--- mod_proxy.c 13 Aug 2004 12:09:13 - 1.130
+++ mod_proxy.c 16 Aug 2004 11:13:35 -
@@ -611,8 +611,14 @@
/* Try to obtain the most suitable worker */
access_status = ap_proxy_pre_request(worker, balancer, r, conf, url);
-if (access_status != OK)
+if (access_status != OK r-method_number != M_CONNECT) {
could connect provide a pre-request hook that does the right thing?
There are different things:
We could be serving to another proxy in this case my patch is just a work-around
and proxy_balancer.c should handle the CONNECT uri and we could have something like:
Proxy 0.0.0.0:8080
BalancerMember localhost:8080
BalancerMember pgtr0327.mch.fsc.net:8080
/Proxy
(That means also to support this syntax).
Or to a normal proxy in this case we don't need the new logic.
if this code has to hard-code a check for M_CONNECT, why not simply
avoid the call to ap_proxy_pre_request() in the first place?
We don't want load balancing for CONNECT.
Re: [AJP] proxy status
Guenter Knauf wrote: Hi, That seems rational to me. The reason for proposing [EMAIL PROTECTED] is so that tomcat-dev'ers wouldn't have to swallow the full bandwidth of [EMAIL PROTECTED] (converse of the problem where they asked anyone in [EMAIL PROTECTED] to follow [EMAIL PROTECTED] for the duration of that proxy_ajp development). hahahahha! I get 5x++ more traffic from tomcat-dev than from [EMAIL PROTECTED] and [EMAIL PROTECTED] together because: - the commit mails go to the same list as the dicussion stuff. - there are so many folks subscribed unable to control their mail server that nearly every day a couple of autoresponder mails come through the list. - there are so many folks subscribed unable to protect their machine from viruses so that every day some viruses come through the list. in addition its anyway a pain with the tomcat-dev list since it drops any attachments which makes it harder for everyone to attach a patch. Reason: the stupid and totally senseless footer. Even with the footer every few days another idiot asks how to unsubscribe - I think that proofes enough that it is useless! So I ask our tomcat-dev'ers who are interested in proxy_ajp, proxy_balancer and so on - are you already subscribed/following [EMAIL PROTECTED] Or do you feel a -strong- need for a lower-traffic list? If no one complains loudly, we will keep all proxy traffic on [EMAIL PROTECTED] (cc's to tomcat-dev if you feel a point needs feedback from the tomcat connector folks.) I'm fine with that. Since the code now moved into httpd HEAD anyway no jtc commiter can commit any more; I f.e. have only karma for the connectors, which means now I can only commit to the old dead code; so for me I see no reason to keep subscribed to tomcat-dev list; the few posts which are really about connectors I can read online, or wait a bit and then crawl though bugzilla when the problems appear there The idea when starting the ajp-proxy was to get it in httpd HEAD. The first developpement were done in j-t-c (jakarta-tomcat-connectors) only because Henri, Mladen and I don't have commit in httpd and we wanted something running for testing very quicky. Now the code is at its right place and we (as Jakarta committers) have to submit patches. (That is what we had done to get the commit rights in Jakarta). my 2ct. Guenter. BTW: also it was asked more than once for a separate section only for connectors in BugZilla - currently you have to do complex searches to get all the connector issues together since they are bound to Tomcat releases, but the connectors are developed independent from Tomcat.
[PATCH] proxy_ajp.c
Hi,
I have arranged the logic to send the request and the body to Tomcat, now it
works for both chunked and not-chunked.
What is not yet working is when the Tomcat starts to send data before having all
the body and then reads a little more body data.
Cheers
Jean-Frederic
Index: proxy_ajp.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/proxy/proxy_ajp.c,v
retrieving revision 1.13
diff -u -r1.13 proxy_ajp.c
--- proxy_ajp.c 11 Aug 2004 23:08:04 - 1.13
+++ proxy_ajp.c 13 Aug 2004 16:11:00 -
@@ -107,6 +107,10 @@
apr_status_t status;
int result;
apr_bucket_brigade *input_brigade;
+ajp_msg_t *msg;
+apr_size_t bufsiz;
+char *buff;
+const char *tenc;
/*
* Send the AJP request to the remote server
@@ -123,51 +127,59 @@
return HTTP_SERVICE_UNAVAILABLE;
}
-/* read the first bloc of data */
-input_brigade = apr_brigade_create(p, r-connection-bucket_alloc);
-status = ap_get_brigade(r-input_filters, input_brigade,
-AP_MODE_READBYTES, APR_BLOCK_READ,
-AJP13_MAX_SEND_BODY_SZ);
-
+/* allocate an AJP message to store the data of the buckets */
+status = ajp_alloc_data_msg(r, buff, bufsiz, msg);
if (status != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
- proxy: ap_get_brigade failed);
-apr_brigade_destroy(input_brigade);
-return HTTP_INTERNAL_SERVER_ERROR;
+ proxy: ajp_alloc_data_msg failed);
+return status;
}
-
-/* have something */
-if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(input_brigade))) {
+/* read the first bloc of data */
+input_brigade = apr_brigade_create(p, r-connection-bucket_alloc);
+tenc = apr_table_get(r-headers_in, Transfer-Encoding);
+if (tenc strcasecmp(tenc, chunked)==0) {
+ /* The AJP protocol does not want body data yet */
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
- proxy: APR_BUCKET_IS_EOS);
-}
-
-if (1) { /* only when something to send ? */
-ajp_msg_t *msg;
-apr_size_t bufsiz;
-char *buff;
-status = ajp_alloc_data_msg(r, buff, bufsiz, msg);
+ proxy: request is chunked);
+} else {
+status = ap_get_brigade(r-input_filters, input_brigade,
+AP_MODE_READBYTES, APR_BLOCK_READ,
+AJP13_MAX_SEND_BODY_SZ);
+
if (status != APR_SUCCESS) {
-return status;
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: ap_get_brigade failed);
+apr_brigade_destroy(input_brigade);
+return HTTP_INTERNAL_SERVER_ERROR;
}
+
+/* have something */
+if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(input_brigade))) {
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
+ proxy: APR_BUCKET_IS_EOS);
+}
+
+/* Try to send something */
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
proxy: data to read (max %d at %08x), bufsiz, buff);
-/* calls apr_brigade_flatten... */
status = apr_brigade_flatten(input_brigade, buff, bufsiz);
if (status != APR_SUCCESS) {
- ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
- proxy: apr_brigade_flatten);
+apr_brigade_destroy(input_brigade);
+ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
+ proxy: apr_brigade_flatten);
return HTTP_INTERNAL_SERVER_ERROR;
}
+apr_brigade_cleanup(input_brigade);
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server,
proxy: got %d byte of data, bufsiz);
if (bufsiz 0) {
status = ajp_send_data_msg(conn-sock, r, msg, bufsiz);
if (status != APR_SUCCESS) {
+apr_brigade_destroy(input_brigade);
ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
- proxy: request failed to %pI (%s),
+ proxy: send failed to %pI (%s),
conn-worker-cp-addr,
conn-worker-hostname);
return HTTP_SERVICE_UNAVAILABLE;
@@ -179,8 +191,9 @@
status = ajp_read_header(conn-sock, r,
(ajp_msg_t **)(conn-data));
if (status != APR_SUCCESS) {
+apr_brigade_destroy(input_brigade);
ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
- proxy: request failed to %pI (%s),
+ proxy: read response failed from %pI (%s),
conn-worker-cp-addr,
conn-worker-hostname);
return HTTP_SERVICE_UNAVAILABLE;
@@
Re: Invitation to HTTPD commiters in tomcat-dev
Graham Leggett wrote: Mladen Turk wrote: I don't think that it is necessary for a mod_ajp to be included inside the mod_proxy, although they are sharing some common concepts. I think it's very necessary - sharing those common concepts ultimately makes for doing things in a consistent way. It makes a big difference to the usability of httpd. Right now proxy is able to talk HTTP and FTP (and CONNECT, but it's a special case). It makes the most sense for AJP to be added to these three protocols, as there is already an established way to do this. Consistency is very important. Having load balancer on top of mod_proxy would be a nice feature, but the main purpose for them is different. Different to what? Load balancing is load balancing, whether the backend protocol is HTTP, AJP or FTP. I see no point on making significant effort in a feature that can only be used for one protocol, that's a huge waste of an opportunity to solve the load balancing problems of backends other than tomcat. The purpose of mod_ajp is to communicate with the (one or more of them in a cluster) application servers using ajp13+ protocol; simple as that. Proxy module has a conceptually different approach, and it is meant to be used for different purposes. I rewrote proxy, so I know - proxy has the exact same conceptual approach and is used for the exact same purposes. Proxy allows you to communicate with (one or more in a cluster) applications servers using HTTP or FTP. The only difference is the protocol. I see in ap_proxy_http_handler() that DECLINED allows to try another. Is there somewhere an example of a configuration using it? The development of proxy_ajp could see the development of modules like proxy_loadbalance or proxy_sticky, which have general application outside of the AJP protocol. Just rewriting mod_ajp for v2.0 isn't anything different to what exists now, so I don't see the point. Regards, Graham --
Re: libtool (jlibtool)
Joe Orton wrote:
On Wed, Jul 07, 2004 at 09:44:00AM +0200, jean-frederic clere wrote:
Joe Orton wrote:
On Tue, Jul 06, 2004 at 03:35:30PM +0200, jean-frederic clere wrote:
The aim is to ensure that the LIBTOOL variable in the installed
config_vars.mk points at the installed copy of the libtool script.
Yes, but I see 3 problems:
- On os2 host configure uses aplibtool, not libtool.
- When using jlibtool the command sh libtool fails.
- shlibtool (in SH_LIBTOOL) is changed into libtool.
- it breaks if not using the bundled APR, since there is no
$(installbuilddir)/libtool then either
To solve those I would use 2 new variables INSTALL_LIBTOOL and
INSTALL_SH_LIBTOOL and fill them in configure by something like:
`basename $LIBTOOL` is known by configure so I wouldn't do that in the
Makefile necessarily. Is there a simpler fix for all four cases by
changing the existing sed substitution on config_vars.mk to something
like:
/LIBTOOL = /s,$(top_builddir)\(srclib/apr\),$(installbuilddir),
Can you try that?
Won't be: /LIBTOOL = /s,$(top_builddir)\(/srclib/apr/\|/\),$(installbuilddir)/,
better?
And we add the $(SHELL) at the configure time if needed, don't we?
Does APR really install {sh,ap,j}libtool in
$installbuilddir already anyway? I can't see where that happens.
The configure of APR replaces libtool by the executable it compiles. (See the
configure.in of APR around line 150)
+++
*-os2*)
# Use a custom-made libtool replacement
echo using aplibtool
LIBTOOL=$srcdir/build/aplibtool
gcc $CFLAGS $CPPFLAGS -o $LIBTOOL.exe $LIBTOOL.c
;;
*)
if test x$LTFLAGS = x; then
LTFLAGS='--silent'
fi
if test $experimental_libtool = yes; then
# Use a custom-made libtool replacement
echo using jlibtool
LIBTOOL=$apr_builddir/libtool
LIBTOOL_SRC=$apr_srcdir/build/jlibtool.c
if test ! -f $LIBTOOL_SRC; then
AC_MSG_ERROR([Experimental libtool source not found. It is not
included with APR by default.])
fi
$CC $CFLAGS $CPPFLAGS -o $LIBTOOL $LIBTOOL_SRC
else
+++
shlibtool is not installed. BTW shlibtool is/was for libtool 1.3.x, isn't?
joe
Re: [PATCH] mod_charset_lite.c
Jeff Trawick wrote:
jean-frederic clere wrote:
Hi,
When using mod_jk2 and mod_charset_lite mod_charset_lite cores because
r-filename is NULL.
patch looks reasonable... unfortunately, I lifted some bad style from
somewhere (the code around your patch)... I'll fix that up and then
commit the appropriate patch
I have to fix a little more in the file because binary files were incorrectly
convert by xlate_in_filter().
Find the new patch enclosed (I have also (tried) to arrange the style around the
patch).
Cheers
Jean-Frederic
if (r-proxyreq) return DECLINED;
+if (r-filename == NULL) return DECLINED;
Index: modules/experimental/mod_charset_lite.c
===
RCS file: /home/cvs/apache/httpd-2.0/modules/experimental/mod_charset_lite.c,v
retrieving revision 1.72
diff -u -r1.72 mod_charset_lite.c
--- modules/experimental/mod_charset_lite.c 29 Jun 2004 13:46:54 - 1.72
+++ modules/experimental/mod_charset_lite.c 9 Jul 2004 16:04:16 -
@@ -225,12 +225,25 @@
}
/* catch proxy requests */
-if (r-proxyreq) return DECLINED;
+if (r-proxyreq) {
+return DECLINED;
+}
+if (r-filename == NULL) {
+return DECLINED;
+}
/* mod_rewrite indicators */
-if (!strncmp(r-filename, redirect:, 9)) return DECLINED;
-if (!strncmp(r-filename, gone:, 5)) return DECLINED;
-if (!strncmp(r-filename, passthrough:, 12)) return DECLINED;
-if (!strncmp(r-filename, forbidden:, 10)) return DECLINED;
+if (!strncmp(r-filename, redirect:, 9)) {
+return DECLINED;
+}
+if (!strncmp(r-filename, gone:, 5)) {
+return DECLINED;
+}
+if (!strncmp(r-filename, passthrough:, 12)) {
+return DECLINED;
+}
+if (!strncmp(r-filename, forbidden:, 10)) {
+return DECLINED;
+}
mime_type = r-content_type ? r-content_type : ap_default_type(r);
@@ -1037,6 +1050,15 @@
dc dc-charset_default ? dc-charset_default : (none));
}
+/* Copy from the xlate_out_filter */
+if (!ctx-noop) {
+const char *mime_type = f-r-content_type ? f-r-content_type :
ap_default_type(f-r);
+if (strncasecmp(mime_type, text/, 5) != 0
+strncasecmp(mime_type, message/, 8) != 0) {
+ctx-noop = 1;
+}
+}
+
if (!ctx-ran) { /* filter never ran before */
chk_filter_chain(f);
ctx-ran = 1;
Re: libtool (jlibtool)
Joe Orton wrote: On Tue, Jul 06, 2004 at 03:35:30PM +0200, jean-frederic clere wrote: I am using jlibtool and I have found something weird in httpd: In Makefile.in: +++ cp build/*.mk $(DESTDIR)$(installbuilddir); \ sed 's#LIBTOOL = \(.*\)#LIBTOOL = $(SHELL) $(installbuilddir)/libtool $(LTFLAGS)#' \ build/config_vars.mk $(DESTDIR)$(installbuilddir)/config_vars.mk; +++ The aim is to ensure that the LIBTOOL variable in the installed config_vars.mk points at the installed copy of the libtool script. Yes, but I see 3 problems: - On os2 host configure uses aplibtool, not libtool. - When using jlibtool the command sh libtool fails. - shlibtool (in SH_LIBTOOL) is changed into libtool. To solve those I would use 2 new variables INSTALL_LIBTOOL and INSTALL_SH_LIBTOOL and fill them in configure by something like: +++ INSTALL_LIBTOOL=$installbuilddir/`basename $LIBTOOL` INSTALL_SH_LIBTOOL=$installbuilddir/`basename $LIBTOOL` +++ and change the name of the variables at installation: LIBTOOL to SRC_LIBTOOL SH_LIBTOOL to SRC_SH_LIBTOOL INSTALL_LIBTOOL to LIBTOOL INSTALL_SH_LIBTOOL to SH_LIBTOOL joe
[PATCH] mod_charset_lite.c
Hi, When using mod_jk2 and mod_charset_lite mod_charset_lite cores because r-filename is NULL. The easy patch is attached. Cheers Jean-Frederic Index: modules/experimental/mod_charset_lite.c === RCS file: /home/cvs/apache/httpd-2.0/modules/experimental/mod_charset_lite.c,v retrieving revision 1.72 diff -u -r1.72 mod_charset_lite.c --- modules/experimental/mod_charset_lite.c 29 Jun 2004 13:46:54 - 1.72 +++ modules/experimental/mod_charset_lite.c 7 Jul 2004 15:49:37 - @@ -226,6 +226,7 @@ /* catch proxy requests */ if (r-proxyreq) return DECLINED; +if (r-filename == NULL) return DECLINED; /* mod_rewrite indicators */ if (!strncmp(r-filename, redirect:, 9)) return DECLINED; if (!strncmp(r-filename, gone:, 5)) return DECLINED;
libtool (jlibtool)
Hi, I am using jlibtool and I have found something weird in httpd: In Makefile.in: +++ cp build/*.mk $(DESTDIR)$(installbuilddir); \ sed 's#LIBTOOL = \(.*\)#LIBTOOL = $(SHELL) $(installbuilddir)/libtool $(LTFLAGS)#' \ build/config_vars.mk $(DESTDIR)$(installbuilddir)/config_vars.mk; +++ In configure.in: +++ my_libtool=`$apr_config --apr-libtool` LIBTOOL=$my_libtool \$(LTFLAGS) libtoolversion=`$my_libtool --version` case $libtoolversion in *1.[[45]]*) SH_LIBTOOL='$(LIBTOOL)' SHLTCFLAGS=-prefer-pic LTCFLAGS=-prefer-non-pic -static ;; *) SH_LIBTOOL='$(SHELL) $(top_builddir)/shlibtool $(LTFLAGS)' SHLTCFLAGS= LTCFLAGS= ;; esac +++ Shouldn't be all the logic in configure.in? Like using LIBTOOL=\$(SHELL) $my_libtool \$(LTFLAGS) when gnu libtool and LIBTOOL=$my_libtool \$(LTFLAGS) otherwise. Cheers Jean-Frederic
Re: Has mod_dir changed between 2.0.46 and 2.0.49 ?
Peter Van Biesen wrote: Hi, I've recently migrated from 2.0.46 to 2.0.49 and I have a strange problem. In 2.0.46, my DirectoryIndex was set to include index.jsp, so if a directory contains a index.jsp, tomcat was contacted to serve this. However, when I do this in 2.0.49 tomcat is contacted even if there's no index.jsp, so where I got a response from autoindex I now get a 404 from tomcat. I removed the index.jsp from my DirectoryIndex and added it everywhere I know there's one, so now it all seems to work, but I'm wondering : is this a bug in the 2.0.46 or in the 2.0.49 ? Or should I change something in my configuration ? What do you use to connect to Tomcat? Thanks in advance ! Peter.
Re: mod_jk / mod_jk2 : help from specialists welcome
Henri Gomez wrote:
Hi to all,
I'm involved in jk/jk2 on tomcat and we wonder on tomcat-dev
if we should use translate in MIDDLE or FIRST position (specifying
that mod_rewrite to be the first in hooks chain).
I see in jk that we're using :
ap_hook_translate_name(jk_translate,NULL,NULL,APR_HOOK_MIDDLE);
and now in jk2 :
static const char * const aszPre[] = { mod_rewrite.c, NULL };
ap_hook_translate_name(jk2_translate, aszPre, NULL, APR_HOOK_FIRST);
What do you think of it ?
MIDDLE or FIRST but after mod_rewrite ?
Thanks for your help and advices !!!
MIDDLE helps to fix 21546... But I do not see why ;-(
signature.asc
Description: OpenPGP digital signature
Re: mod_jk / mod_jk2 : help from specialists welcome
Justin Erenkrantz wrote: --On Wednesday, March 3, 2004 10:16 AM +0100 jean-frederic clere [EMAIL PROTECTED] wrote: MIDDLE helps to fix 21546... But I do not see why ;-( Oh, *now* I understand the context to Henri's question. Here's a suggestion. And, no, I don't think changing the priority to MIDDLE (or LAST) will solve the problem either. I think you want to remove the translate_name hook entirely and fold that functionality into the map_to_storage hook. mod_jk isn't translating or modifying r-filename, but instead mapping the path (as determined by all of the other modules) to storage. At the point when translate_name is called, we don't have any idea what path the JSP is going to be at. PR 21546 is causing a problem because mod_jk2 is interfering with the other modules from performing their translation of r-filename. I think the missing module in J-F Clere's analysis of 21546 is mod_alias. mod_alias also has a MIDDLE translate name hook. So, if you switch mod_jk2 to MIDDLE, mod_alias's hook would run before mod_jk2's translate name hook. Since translate_name is a RUN_FIRST hook, it then won't execute mod_jk2's translate name hook. Yet, this would allow mod_dav to operate properly on non-GET/POST requests as the filename will be the 'correct' one which is resolvable by mod_dav_fs. But, what happens when mod_jk's translate hook doesn't run but the Alias hook does? As I read the mod_jk2 code, I don't see how the uriEnv will be setup properly in the r-request_config vector. I have a hunch that mod_jk2 will subsequently pass on all requests when an Alias in effect for that location space if you use MIDDLE for mod_jk2. That's bad. Very bad!: +++ [Wed Mar 03 12:09:04 2004] [error] [client 172.25.182.35] File does not exist: /examples [Wed Mar 03 12:09:05 2004] [notice] child pid 5865 exit signal Segmentation fault (11), possible coredump in /home/apache20/apache20 +++ I have just add: Alias /myniceexamples /examples in httpd.conf that has the mod_jk2 directive: +++ Location /examples JkUriSet worker ajp13:localhost:8009 /Location +++ By moving the code currently in mod_jk2's translate_name to the map_to_storage looks like the correct solution to me. My hunch is that should work then... HTH. -- justin signature.asc Description: OpenPGP digital signature
MODULE_MAGIC_NUMBER_MAJOR complain
Hi, Would there be a way to get ride of the following problem: +++ httpd: module ../../server/apache2/mod_jk2.c is not compatible with this version of Apache (found 20020903, need 20030213). Please contact the vendor for the correct version. +++ In include/ap_mmn.h we have: +++ * 20020612 (2.0.38-dev) Changed server_rec-[keep_alive_]timeout to apr time * 20020625 (2.0.40-dev) Changed conn_rec-keepalive to an enumeration * 20020628 (2.0.40-dev) Added filter_init to filter registration functions * 20020903 (2.0.41-dev) APR's error constants changed +++ I understand very well that a module compiled for the 2.0.45 will not run on 2.0.39 but I am quite disappointed not to be able to run a module compiled in 2.0.39 in the 2.0.45. Cheers Jean-frederic
Re: MODULE_MAGIC_NUMBER_MAJOR complain
Jeff Trawick wrote: jean-frederic clere wrote: Hi, Would there be a way to get ride of the following problem: +++ httpd: module ../../server/apache2/mod_jk2.c is not compatible with this version of Apache (found 20020903, need 20030213). Please contact the vendor for the correct version. That looks like a change to MMN between 2.0.41 and 2.1-dev. Expect it. In include/ap_mmn.h we have: +++ * 20020612 (2.0.38-dev) Changed server_rec-[keep_alive_]timeout to apr time * 20020625 (2.0.40-dev) Changed conn_rec-keepalive to an enumeration * 20020628 (2.0.40-dev) Added filter_init to filter registration functions * 20020903 (2.0.41-dev) APR's error constants changed +++ I understand very well that a module compiled for the 2.0.45 will not run on 2.0.39 but I am quite disappointed not to be able to run a module compiled in 2.0.39 in the 2.0.45. It is a little bit late to complain about 2.0.39 modules not working with 2.0.41+. Agreed. BTW: 20030213 is from 2.1.0-dev. 2.0.39 versus 2.0.41+ tells (found 20020612, need 20020903) ;-) 2.0.45 will be 20020903, won't it? At any rate, pain like this helped comfirm the need for a stable 2.0 tree to maintain without breaking module compatibility. Sure.
Re: Solaris 8 + gcc 3.2.1 + libtool = fuckup...
Pier Fumagalli wrote: Again, here goes my little problem :-( I'm trying to compile 2.0.43 on a Solaris 8.0 (latest patches) machine, using gcc-3.2.1 (which I compiled) and the libtool coming in the dist (no, I didn't re-run buildconf), and I have one problem. Let's start by saying that my gcc is compiled with --enable-static --disable-shared (which is going to make things complicated), so libgcc is only available as libgcc.a (no libgcc.so). As I compile on a development machine, and deploy on another, I don't want to bring libgcc.so around, so, with everything else, it works... Now, the problem is the following: when I compile httpd, and try to invoke httpd -t, it'll simply refuse to load saying: Cannot load /opt/apache/modules/mod_status.so into server: ld.so.1: /opt/apache/bin/httpd: fatal: relocation error: file /opt/apache/modules/mod_status.so: symbol __floatdisf: referenced symbol not found This also happen with mod_auth_digest.so... Tracing down the problem, it seems that libgcc defines __floatdisf, but the ./configure process never figures out that it'll need to link certain modules against it. The odd thing (odd, well, its ANOTHER libtool bug) is that when I specify LDFLAGS=-lgcc to configure, all modules are created as static libraries (because libtool cowardly refuses to link something against a static and a shared library)... Now, gcc _correctly_ links any executable to the correct library libgcc.a automagically (if I call it from the command line), _but_ libtool is stupid, and assumes that when we want to link a library, we can simply call ld automatically without caring about what GCC might have to say... And then I get into a deadlock... To compile mod_status and mod_auth_digest, I need to use libtool, but libtool calls ld directly without the inclusion of libgcc, so, basically, I'm foobared... My quick-and-dirty solution was to call LD myself adding libgcc on the command line and manually re-link the two modules, but I bet that I'm not the only one who will get into this problem as HTTPD-2.0 gets deployed more widely (this doesn't happen with 1.3). I haven't tried with Justin's jlibtool (as it doesn't want to work against the latest 2.0.43, darn) but if someone wants to do what I did, and use libgcc static, is going to run into troubles... Now, I tried to do two things: link httpd including libgcc and adding libgcc.a to libapr... They both don't work, so, at the end, I'll have to hack every module depending on something defined by libgcc (apxs won't work for those modules)... Anyone got a friggin' CLUE? Throw me a bone out here! :-) :-) BTW, if someone wants an account on the troublesome set of machines, just let me know... :-) I remember trying gnu ld instead of native ld. I have just tried to build gcc 3.2.1 (and tell him to use native ld) on my machine but it fails: +++ /export/home2/jfclere/gcc-3.2/libjava/prims.cc:577: `__builtin_va_arg_incr' undeclared (first use this function) gmake[5]: *** [prims.lo] Error 1 gmake[5]: Leaving directory `/export/home2/jfclere/gcc/sparc-sun-solaris2.8/spar +++ Does this sound to someone? Pier
Re: Whopsie on Darwin/Mac OS X 10.2.1... :-(
Pier Fumagalli wrote: On 7/10/02 21:45, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: At 03:27 PM 10/7/2002, Sander Temme wrote: Your HEAD probably uses the glibtool(ize) installation on your local box, which on 10.2 by default is 1.4.2. The tarball was built using the FreeBSD libtool, which is 1.3.4. This version did not know about Darwin yet and will not create any sort of shared library on this platform. The solution is building the tarball with a more recent version of libtool. Maybe Apache should fail more conclusively if the user wants .so modules and the build system can't do them, but that's a different question from getting the functionality to work. I think the ASF roll environment should bump its libtool. I doubt Darwin is the only platform that would benefit from that. This is my doing. Suggestion; could you offer a patch to build/httpd_roll_release that warns the RM that the version of buildconf is too stale? Checking against 1.4.2 would be a good-thing(TM) indeed, but doesn't guarantee that on certain platforms (such as darwin, where the mainstream libtool port doesn't work) this will not break things again... We've been playing the libtool game since I started building 2.0. At one point or another, it broke things (I remember AIX as well), and as far as I know, noone has ever been able to get a patch incorporated into the main tree (I mean, removing a couple of is not a big deal, right?)... We can't keep libtool on our CVS as it's GPLed, let's just keep it off somewhere, apply the patches _we_ need, and keep our machines updated with _our_ version which works _for_us_... Right? And yes - updating all the Apache machines would be convienent. If my ex-girlfriend decides to give me back my Cube, I might be able to keep it as an Apache Server for development and Testing on MacOS/X, since it seems that moof is completely unmaintained... I have had to fight a little to get httpd-2.0.43 compiled and running on moof. And I am looking for a JDK1.3 for it. Any hints? Pier
Adding a module to httpd-2.0
Hi, I am adding mod_jk (from Jakarta Tomcat) to the httpd-2.0 sources in order to build an httpd excutable with mod_jk linked staticly to it. But I have to make a buildconf to integrate the config.m4 in the configure. Is there a better way to get it? Like: --activate-module=src/modules/jk/libjk.a --disable-shared=jk we are using with Apache-1.3 I have in modules/jk: config.m4 lib_jk.a lib_jk.la But I think I have to add mod_jk.c and some include to compile it. Cheers Jean-frederic
Re: cvs commit: apache-1.3/src/ap ap_strtol.c Makefile.tmpl
David McCreedy wrote: I agree with this approach. I've confirmed that TPF's strtol function returns an errno of ERANGE. I don't know if OS390's and BS2000's compilers do as well. The strtol() of the BS2000 also returns ERANGE (2). Jim has updated ap_strtol.c with an explicit warning: ... this will break on EBCDIC machines if base is 19. And since Apache itself only does base 10 and base 16 I'm not going to do a last minute patch for TPF/EBCDIC. It's not worth the risk of complicating or breaking the T+R of 1.3.25. -David Jim Jagielski [EMAIL PROTECTED]To: [EMAIL PROTECTED] (Justin Erenkrantz) cc: [EMAIL PROTECTED], [EMAIL PROTECTED] 06/17/2002 09:14 Subject: Re: cvs commit: apache-1.3/src/ap ap_strtol.c Makefile.tmpl PM Please respond to dev +1 Justin Erenkrantz wrote: On Mon, Jun 17, 2002 at 11:06:30PM -0400, Jim Jagielski wrote: With 2.0, we are lucky because we require not only an ANSI compiler but also the full ANSI function suite. That's never been the case on 1.3. So, at least with 1.3, we can't assume that strtol() works as we require, since we don't require strict ANSI functionality. We could have done a NEED_STRTOL, but that would require more knowledge than we have right now, I'm afraid. I guess my point was that we could use the system strtol() for EBCDIC machines on 1.3 to handle this special case. Since those machines are typically confined to one particular compiler and library, we can easily verify that those strtol() implementations are correct. #ifdef CHARSET_EBCDIC #define ap_strtol strtol #else ...whatever we have now based on the BSD code... #endif Just a thought. -- justin -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: tarballs are up for testing
Cliff Woolley wrote: ... in http://httpd.apache.org/dev/dist/ --Cliff I have noted the following on ReliantUnix: +++ Installing configuration files Installing HTML documents Usage: rm [-firR] file ... make[1]: *** [install-htdocs] Error 2 make[1]: Leaving directory `/home1/apache20/httpd-2.0.37' make: *** [install-recursive] Error 1 +++ The fix is easy. Find it attached. Cheers Jean-frederic --- Makefile.in.org Wed Jun 12 15:53:19 2002 +++ Makefile.in Wed Jun 12 15:54:18 2002 -108,25 +108,25 test -d $(htdocs-srcdir) (cd $(htdocs-srcdir) cp -rp * $(htdocsdir)) test -d $(manualdir) || $(MKINSTALLDIRS) $(manualdir) test -d $(top_srcdir)/docs/manual (cd $(top_srcdir)/docs/manual cp -rp * $(manualdir)) - test x$(htdocsdir) != x cd $(htdocsdir) find . -name CVS -print | xargs rm -rf + - test x$(htdocsdir) != x cd $(htdocsdir) find . -name CVS -print +| xargs rm -rf install-error: echo Installing error documents test -d $(errordir) || $(MKINSTALLDIRS) $(errordir) cd $(top_srcdir)/docs/error cp -rp * $(errordir) - test x$(errordir) != x cd $(errordir) find . -name CVS -print | xargs rm -rf + - test x$(errordir) != x cd $(errordir) find . -name CVS -print | +xargs rm -rf install-icons: echo Installing icons test -d $(iconsdir) || $(MKINSTALLDIRS) $(iconsdir) cd $(top_srcdir)/docs/icons cp -rp * $(iconsdir) - test x$(iconsdir) != x cd $(iconsdir) find . -name CVS -print | xargs rm -rf + - test x$(iconsdir) != x cd $(iconsdir) find . -name CVS -print | +xargs rm -rf install-cgi: echo Installing CGIs test -d $(cgidir) || $(MKINSTALLDIRS) $(cgidir) cd $(top_srcdir)/docs/cgi-examples cp -rp * $(cgidir) - test x$(cgidir) != x cd $(cgidir) find . -name CVS -print | xargs rm -rf + - test x$(cgidir) != x cd $(cgidir) find . -name CVS -print | +xargs rm -rf install-other: test -d $(logfiledir) || $(MKINSTALLDIRS) $(logfiledir)
Re: 2.0.37-dev/Solaris-8/sparc-v9
Jeff Trawick wrote: Pier Fumagalli [EMAIL PROTECTED] writes: I'm about to attempt to build Apache 2.0.37-dev at 64 bits with the Sun C compiler (kindly donated by Sun), on Nagoya (worker MPM). Anything I should be aware of? I'll report back on how the baby flies (if it does)... There is an open PR about a 64-bit build on Solaris. IIRC, the atomic stuff adds some v8plus flag that the linker doesn't like. See PR 9247. That is with gcc3.1, it works with gcc3.0.4 Should we fix the problem? I thought atomics may be removed.
Re: 2.0.37-dev/Solaris-8/sparc-v9
Ian Holsman wrote: jean-frederic clere wrote: Jeff Trawick wrote: Pier Fumagalli [EMAIL PROTECTED] writes: I'm about to attempt to build Apache 2.0.37-dev at 64 bits with the Sun C compiler (kindly donated by Sun), on Nagoya (worker MPM). Anything I should be aware of? I'll report back on how the baby flies (if it does)... There is an open PR about a 64-bit build on Solaris. IIRC, the atomic stuff adds some v8plus flag that the linker doesn't like. See PR 9247. That is with gcc3.1, it works with gcc3.0.4 Should we fix the problem? I thought atomics may be removed. pr 9247 is a bug in the configure section. we don't check to see if we are in 64bit mode, we always assume 32bit mode... and sun's compiler needs a 'v9' sent to it to build the assembler in 64bit mode.. Yep: +++ $ cc -o toto toto.c $ file ./toto ./toto: ELF 32-bit MSB executable SPARC Version 1, dynamically linked, not stripped +++ And: +++ $ cc -xarch=v9 -o toto toto.c $ file ./toto ./toto: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically linked, not stripped +++ With gcc: +++ $ gcc -Wa,-xarch=v9 -o toto toto.c /opt/SMAW/gnu/lib/gcc-lib/sparc-sun-solaris2.8/3.0.4/../../../../sparc-sun-solaris2.8/bin/ld: warning: sparc:v9 architecture of input file `/var/tmp/ccSz3T8a.o' is incompatible with sparc output $ gcc -Wa,-xarch=v8plus -o toto toto.c $ file ./toto ./toto: ELF 32-bit MSB executable SPARC Version 1, dynamically linked, not stripped +++ What is wrong? BTW.. you do know that 64bit programs take a ~10% hit in performance don't you? Only? --ian
Re: Apache History Project - Call for comments
Bill Stoddard wrote: Ben Hyde wrote: Thomas Eibner wrote: One volunteer has already come forward to help us gather data and we've compiled some statistics from this mailinglist showing how many posts each person has made since '95: Total Posts on [EMAIL PROTECTED]: 87510 1: ... This comes up from time to time. Somebody measures something; bugs submitted, bugs fixes, lines written, commits made, documents written, mails send, bylaws written, meetings attended, email threads started, email threads ended, uses of the word 'bucket'... When it has come up before I have written variations of the following. Try not to take this as an attack on the volunteer or the messenger but ... I have very strong objections to nieve attempts to reduce the contributions made by various individuals into 'objective' scores and 'leader boards'. I agree as well... such metrics are misleading at best and insulting and disruptive at worst. They are, to be blunt, dangerous to a collaborative environment and should be avoided if possible. -- Yes, I agree with Jim and Ben. If this is where the project is going, I am in favor of shutting it down right now. Bill Such metrics are like counting the number of commits per committer in a projet: Funny but senseless! Ranking in the mail could be worse... Flames are (mostly) unless but are counting there. May be an interesting thing would be the ranking of answering to user questions in user list by committers.
Re: 404 .var file served as text/plain revisited
Cliff Woolley wrote: I know this has been brought up before, but what did we decide was the fix for this problem? I have a test install that's almost identical to httpd-std.conf, and yet my 404 .var file gets served as text/plain. HEAD /server-status/ HTTP/1.0 Host: localhost HTTP/1.1 404 Not Found Date: Mon, 18 Mar 2002 21:32:18 GMT Server: Apache/2.0.34-dev (Unix) mod_ssl/3.0a0 OpenSSL/0.9.6c DAV/2 Vary: accept-language Content-Length: 790 Connection: close Content-Type: text/plain; charset=ISO-8859-1 [discovered because I neglected to configure mod_status ;] Thanks, --Cliff -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA I have so more funny things related to the topic today: +++ HEAD /toto/ HTTP/1.0 Host: vtxrm2 HTTP/1.1 501 Method Not Implemented Date: Wed, 20 Mar 2002 15:06:14 GMT Server: Apache/2.0.34-dev (Unix) DAV/2 Allow: GET,HEAD,POST,OPTIONS,TRACE Content-Length: 326 Connection: close Content-Type: text/html; charset=iso-8859-1 +++ That is without mod_include (with mod_include I get the text/plain 501*.var file). Cheers Jean-frederic
Re: Latest CVS on Solaris 8 - configure's cache-file problems
Jeff Trawick wrote: Jim Jagielski [EMAIL PROTECTED] writes: Yep... Looks like it's no problem with 2.52. Anyone have hearburn if I adjust buildcheck.sh to make 2.52 the new requirement? at least a little, but I'm not exactly sure how much :) In the 2.0.30 timeframe a colleage was unable to get make distclean followed by a new ./configure configure to work on AIX when using autoconf 2.52. Backing down to autoconf 2.13 resolved the problem. But maybe that has been fixed problem has subsequently been fixed (some stray file left around?). I didn't see an indication of which autoconf version you had problems with. I just did cvs update and ./configure on Solaris 8 using autoconf 2.13 and so far it has gotten past the APR configure. Did configure ever work for you on that machine with autoconf 2.52? (I guess what I'd like to say is that no autoconf is perfect -- or perfectly supported by Apache+APR -- and that maybe some debugging is appropriate before we start disallowing a very common version of autoconf.) On ReliantUnix autoconf 2.52 does not work. ./buildconf gives the following error: +++ nawk: Tempcell list is curdled nawk: Source line number 7 +++ 2.13 works well. -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: FYI: worker not running on Solaris 8
Jeff Trawick wrote: jean-frederic clere [EMAIL PROTECTED] writes: Pier Fumagalli wrote: Pier Fumagalli [EMAIL PROTECTED] wrote: Not even starting up... Once I launch it, the first child dies, and no workers are left to process requests... Now trying with threaded... Great... Should RTFM before complaining :) Setting the group to nobody made the trick! :) You are lucky - My httpd-2.0 hangs in getaddrinfo from apr_sockaddr_info_get, the getaddrinfo does 'strange' things with threads - Hey, look at the description of this patch: http://sunsolve.sun.com/pub-cgi/findPatch.pl?patchId=109326rev=05 One of the fixes is: 4500573 Multithreaded applications block in DNS Name Service switch backend. I have ended installing the Solaris recommended patch cluster now httpd-2.0 seems to work correctly. -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: LibTool not working on OS/X?
Pier Fumagalli wrote: Pier Fumagalli [EMAIL PROTECTED] wrote: Anyone seen this before? Libtool 1.4.2 doesn't work on OS/X and the modules produced with Apache 2.0 are actually DYLIB(s) and not MH_BUNDLE(s), thus preventing the APR routines to read them... There seems to be a screwup with glibtool's -module parameter... Gotcha... Do a search for xyes and you will see something like archive_cmds=\$nonopt \$(test \\x\$module\\ = xyes echo -bundle... This has to be exchanged into archive_cmds=\$nonopt \$(test \x\$module\ = xyes echo -bundle... To make it work... Any chance we can do something with it? :) Fix it in ltconfig and send the patch to [EMAIL PROTECTED] Pier
Re: 2.0.32 has been released.
Hi, I have done the configure: ./configure --prefix=/export/home/apache20/apache32 \ --enable-mods-shared=all It fails: +++ configure:3147: gcc -c -g -O2 -pthreads -DSOLARIS2=8 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -I/export/home/apache20/httpd-2.0.32/srclib/apr/include -I/export/home/apache20/httpd-2.0.32/srclib/apr/include conftest.c 15 configure:3142:20: #error You need APR random support to use auth_digest. +++ May be auth_digest should be disabled... Cheers Jean-frederic Justin Erenkrantz wrote: I received 3 +1s to make JRE_1 .32, so it became 2.0.32 (with a CVS tag of APACHE_2_0_32). http://www.apache.org/dist/httpd/httpd-2.0.32-alpha.tar.gz Committers and non-committers, please vote for beta from this tarball. Once it is approved as a beta, the formal release notes will be made with appropriate posts to current-testers and other places as required. Since I need 3 +1s for .32 to call it beta, I placed it up as an alpha (2 +1s so far for beta). I'm certainly hopeful that this will be a beta shortly. (If you dare vote -1 and you didn't test JRE_1 when I asked for feedback, I will come after you with a baseball bat.) This build has been running on daedalus for over three days now (with one minor tweak that is in this release for mod_negotiation), so I believe it has passed our dog food test, but Greg Ames has the final word on daedalus's vote. Some random notes: - autoconf 2.13 and libtool 1.4.2 were used to generate the .gz tarball. IMHO, if this combination doesn't work on your platform, then you need to run ./buildconf yourself. I don't particularly care to roll tarballs for every oddball platform. - I hacked httpd_roll_release to leave in ., so the tarball produces httpd-2.0.32 directory when extracted. - Because I wanted libtool 1.4.2 for the .tar.gz, I rolled it on my local Linux box. However, that box doesn't have compress, so I couldn't post .Z files. If someone needs them, I'm hoping someone can volunteer to post a good .Z file. (The same goes for Win32 and .zip files.) - Since I can not recommend .31 now (and I believe none of us can either), I removed those files from /dist/httpd/ since it now has been obsoleted by .32. I hope this is/was correct policy. - I've tested this with httpd-test and all the tests pass (even negotiation suprisingly enough!). (I have heard sporadic reports of failures with httpd-test, but I can *not* reproduce any of them.) Enjoy. -- justin
Re: Solaris8 Problem with native compiler
Jeff Trawick wrote: Jeff Trawick [EMAIL PROTECTED] writes: 2) POD issue with worker MPM: [Tue Feb 12 12:01:46 2002] [notice] SIGUSR1 received. Doing graceful restart [Tue Feb 12 12:01:46 2002] [warn] (128)Network is unreachable: connect to listener This particular Solaris box has a screwy IPv6 setup which I don't yet know how to fix. For some reason I can get an IPv6 socket and bind to in6addr_any but IPv6 loopback (lo0, ::1) is disabled and I can't connect to :: or ::1 :( The wonderful Sun admin manuals say to simply touch /etc/ifname.6 or something silly like that and then reboot, but still no happiness. Hopefully comp.unix.solaris can offer some guidance. I don't know whether or not the setup problem was present before installing the current recommended patch set. A little more caffeine helped... A re-read of the manual, a re-touch of the magic files, and a re-boot of the system fixed the IPv6 setup and now I don't have the Network is unreachable: connect to listener failure in the log. That is why I was not able to reproduce it, but I had no time to try hard ;-) ifconfig -a shows: +++ lo0: flags=1000849UP,LOOPBACK,RUNNING,MULTICAST,IPv4 mtu 8232 index 1 inet 127.0.0.1 netmask ff00 hme0: flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500 index 2 inet 172.25.124.140 netmask ff00 broadcast 172.25.124.255 ether 0:80:17:28:61:7b lo0: flags=2000849UP,LOOPBACK,RUNNING,MULTICAST,IPv6 mtu 8252 index 1 inet6 ::1/128 hme0: flags=2000841UP,RUNNING,MULTICAST,IPv6 mtu 1500 index 2 ether 0:80:17:28:61:7b inet6 fe80::280:17ff:fe28:617b/10 +++ And I am able to telnet ::1 and fe80::280:17ff:fe28:617b -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: Solaris8 Problem with native compiler
Jeff Trawick wrote: jean-frederic clere [EMAIL PROTECTED] writes: Hi, I have some problems with the httpd-2.0 (from CVS). Whe compiled with Sun compiler it hangs at startup: +++ $c (some internal library calls omitted) libsocket.so.1`getaddrinfo+0x524(ff31ac0c, 93c08, 1e62, ffbefa1c, ff31a000, ffbef980) libapr.so.0`apr_sockaddr_info_get+0x14c(d7cc4, 93c08, 0, 1e62, 0, 95930) ap_mpm_pod_open+0xd8(95930, 8a66c, 49, 1, ff3e, ff370330) prefork_open_logs+0xb0(95930, bd9f8, bfa00, 975e0, bfa00, 0) ap_run_open_logs+0x98(95930, bd9f8, bfa00, 975e0, 0, 0) main+0x864(1, ffbefc9c, ffbefca4, 87800, 0, 0) _start+0xb8(0, 0, 0, 0, 0, 0) At the moment I'm not sure why that apr_sockaddr_info_get() is there. It looks a little mysterious to me, but I haven't looked into it. As for why getaddrinfo() is hanging... It shouldn't do that :) Are there some resolver timeouts you can set on the system? There isn't a WAIT_FOREVER flag to getaddrinfo(), so that is a Sun question :( Back to reality, where we need to deal with this: 1) If you want to experiment with what might work around the problem... You could replace APR_UNSPEC in ap_mpm_pod_check() with AF_INET and see if that avoids the apparent WAIT_FOREVER path in the resolver. In ap_mpm_pod_open() It helps ;-)). 2) Maybe Jeff needs to learn about that particular apr_sockaddr_info_get() call to see if it could be changed. For now, I dunno. I have configured with --disable-threads it works. It seems that getaddrinfo does not like the -mt (and libthreads.so). With gcc (that uses -pthreads) I have the same problem. libnls.so.1 uses threads: +++$ nm /usr/lib/libnsl.so.1 | grep mutex [3514] | 0| 0|FUNC |GLOB |0|UNDEF |_mutex_held [3933] | 0| 0|FUNC |GLOB |0|UNDEF |_mutex_init [3474] | 0| 0|FUNC |GLOB |0|UNDEF |_mutex_lock [3458] | 0| 0|FUNC |GLOB |0|UNDEF |_mutex_unlock [3625] | 0| 0|FUNC |GLOB |0|UNDEF |mutex_destroy [4419] | 0| 0|FUNC |GLOB |0|UNDEF |mutex_init [3970] | 0| 0|FUNC |GLOB |0|UNDEF |mutex_lock [1747] |649784| 80|OBJT |LOCL |0|19 |mutex_table [4070] | 0| 0|FUNC |GLOB |0|UNDEF |mutex_unlock [331] |648984| 24|OBJT |LOCL |0|19 |netpp_mutex [1718] |295600| 40|FUNC |LOCL |0|9 |rmutex_init [1719] |295640| 132|FUNC |LOCL |0|9 |rmutex_lock [1720] |295772| 116|FUNC |LOCL |0|9 |rmutex_trylock [1721] |295888| 64|FUNC |LOCL |0|9 |rmutex_unlock [454] |650328| 24|OBJT |LOCL |0|19 |rpc_door_mutex [1959] |650184| 24|OBJT |LOCL |0|19 |rpcgss_calls_mutex [240] |650280| 24|OBJT |LOCL |0|19 |svc_door_mutex [227] |681576| 24|OBJT |LOCL |0|22 |svc_exit_mutex [562] |694312| 24|OBJT |LOCL |0|22 |svc_mutex [152] |681344| 24|OBJT |LOCL |0|22 |svc_thr_mutex +++ --/-- For better or worse, some months back APR decoupled the choice of using getaddrinfo() from the choice of enabling IPv6 support, so --disable-ipv6 doesn't switch us to the older, better-understood resolver logic. -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Solaris8 Problem with native compiler
Hi, I have some problems with the httpd-2.0 (from CVS). Whe compiled with Sun compiler it hangs at startup: +++ $c libc.so.1`_lwp_sema_wait+8(8d7f0, ff19e000, 0, 8d738, 2501c, 0) libthread.so.1`_swtch+0x400(8d738, 8d738, ff19e000, 5, 671, cf3c0) libthread.so.1`_mutex_adaptive_lock+0x144(ff1a9944, 6400, 4c00, 1, 4d58, fffe) libthread.so.1`_cmutex_lock+0x50(ff29e448, ff, ff280fce, ff21a82c, ffbef680, ff280fce) libnsl.so.1`_get_hostserv_inetnetdir_byname+0x224(ffbef78c, ffbef794, 0, ffbef794, ff29ae30, 93c09) libnsl.so.1`getipnodebyname+0x12c(0, 0, 8e6a8, ffbef80c, 93c08, cea38) libsocket.so.1`get_addr+0xc4(0, 93c08, ffbef980, 1, ff31a000, 1e62) libsocket.so.1`getaddrinfo+0x524(ff31ac0c, 93c08, 1e62, ffbefa1c, ff31a000, ffbef980) libapr.so.0`apr_sockaddr_info_get+0x14c(d7cc4, 93c08, 0, 1e62, 0, 95930) ap_mpm_pod_open+0xd8(95930, 8a66c, 49, 1, ff3e, ff370330) prefork_open_logs+0xb0(95930, bd9f8, bfa00, 975e0, bfa00, 0) ap_run_open_logs+0x98(95930, bd9f8, bfa00, 975e0, 0, 0) main+0x864(1, ffbefc9c, ffbefca4, 87800, 0, 0) _start+0xb8(0, 0, 0, 0, 0, 0) +++ Any hints? Cheers Jean-frederic
Re: daedalus is running 2.0.31
Ian Holsman wrote: Greg Ames wrote: ...since Thursday, 31-Jan-2002 19:04:06 PST. Cool. Something hangs now on daedalus! we're running it for our developers internally starting tomorrow. Beside checking out the tag, it has the usual patch to save the input buffers for debugging, and a quick-n-dirty hack to exit the child without killing the parent if accept() gets ENFILE (system out of fd's). I did have to futz with the config file a bit. The main thing was the change from mod_auth_db to mod_auth_dbm. I hope the following is correct, please holler if not: @@ -762,7 +764,8 @@ Options All /Directory Directory /da1/www/bugs.apache.org/private - AuthDBUserFile /home/apmail/bugdbaccounts + AuthDBMUserFile /home/apmail/bugdbaccounts + AuthDBMType DB your running berkeleyDB ??? if so that is the right config. AuthName ApacheBugDatabaseUsers AuthType Basic require valid-user Are there any changes to the utility to manage the passwords that Brian and Manoj should know about? htdbm has been changed as well so as to allow for multiple DB types I think it is -T DB as an extra parameter if you don't have berkeleyDB support as default. --Ian Greg
