Re: Sharing information across Apache child processes
On 09/29/2014 05:14 PM, Sorin Manolache wrote: On 2014-09-29 13:39, Rajalakshmi Iyer wrote: Hello, I have a requirement whereby my application's configuration information (comprising a few complex data structures) needs to be shared across the various Apache child processes. Currently, the configuration is being individually loaded by each child process, which makes it hard for configuration changes to propagate. What is the best way / place to have a common configuration for the application? Please advise. I suppose you want to update the configuration without running apache2 -k graceful (or apache2ctl graceful). In this case you could use a segment of memory that is shared across the apache children. You'll have to create the shared segment memory before the parent forks its children (for example in post_config). The shared memory is then inherited by the forked children. You'll need a method to update the contents of the shared memory segment and a multiple-readers-single-writer inter-process exclusion mechanism in order to safely read and write from the shared segment. Sorin Hello All, You can check out mod_cluster approach (https://github.com/modcluster/mod_cluster/tree/master/native/mod_proxy_cluster) - it uses shared memory as data storage and additional thread per process to check updates in storage. This way you'll keep per-process module config in sync with shared data. Not perfect solution though, but I doubt that there is any flawless way to share config structures. Best regards, Abdi A.
Re: Sharing information across Apache child processes
There's a variation on this theme you might consider. You can use a few counters in shared memory, and save config information in a file. If one of the child processes learns of a config update (say by handling an HTTP request for one from somewhere) it can update the file (atomically, say, via write-to-temp-and-rename) and bump a shared-memory counter (atomically via shared-memory mutex). Other child processes can quickly poll on counter-changes and re-read config files when they change. The advantage of this approach over storing the whole config in shared-memory is that we only put a fixed number of integers in shared memory, rather than our whole configuration structure, which might be large or vary in size depending on the config. mod_pagespeed uses this technique https://code.google.com/p/modpagespeed/source/browse/trunk/src/pagespeed/kernel/cache/purge_context.h. Note there is no apr code directly in that file as it's abstracted behind some C++ interfaces, but 'Variable' is really an int64's worth of mutexed shared memory in operation. -Josh - On Tue, Sep 30, 2014 at 9:20 AM, Abdi Abdirahman abd.moha...@gmail.com wrote: On 09/29/2014 05:14 PM, Sorin Manolache wrote: On 2014-09-29 13:39, Rajalakshmi Iyer wrote: Hello, I have a requirement whereby my application's configuration information (comprising a few complex data structures) needs to be shared across the various Apache child processes. Currently, the configuration is being individually loaded by each child process, which makes it hard for configuration changes to propagate. What is the best way / place to have a common configuration for the application? Please advise. I suppose you want to update the configuration without running apache2 -k graceful (or apache2ctl graceful). In this case you could use a segment of memory that is shared across the apache children. You'll have to create the shared segment memory before the parent forks its children (for example in post_config). The shared memory is then inherited by the forked children. You'll need a method to update the contents of the shared memory segment and a multiple-readers-single-writer inter-process exclusion mechanism in order to safely read and write from the shared segment. Sorin Hello All, You can check out mod_cluster approach (https://github.com/ modcluster/mod_cluster/tree/master/native/mod_proxy_cluster) - it uses shared memory as data storage and additional thread per process to check updates in storage. This way you'll keep per-process module config in sync with shared data. Not perfect solution though, but I doubt that there is any flawless way to share config structures. Best regards, Abdi A.
Re: How to wait on a global lock with timeout
Hi Micha, On Mon, Sep 29, 2014 at 3:18 PM, Micha Lenk mi...@lenk.info wrote: in an Apache module I am in the need to wait for a global lock (e.g. an apr_global_mutex_t), but in theory the lock might not get released by the other process in a timely manner, so I would like to limit the time to wait for the lock. What do you suggest me to do? I have been working on a patch to provide apr_[thread/proc]_mutex_timedlock() in APR, but did not finish the work mostly because of APR_ENOTIMPL on some mutex mechanisms (mainly Windows CRITICAL_SECTIONs which lack the functionality, making the APR abstraction quite useless IMO, or at least unix specific). Maybe I'll have a new look at it these days if I have the time to. For now you may have the possibility to use native mutex functions (eg. pthread_mutex_timedlock or WaitForSingleObject) with the native object retrieved by ap_os_[thread/proc]_mutex_get(), but this requires either to create the mutex with a mechanism supporting the timeout, or the existing mutex to have such a mechanism. Hence this may be non-portable code, or with many APR_HAS_*_SERIALIZE, and still the problem with platforms that don't support it... Regards, Yann.
Re: How to wait on a global lock with timeout
On Tue, Sep 30, 2014 at 5:30 PM, Yann Ylavic ylavic@gmail.com wrote: I have been working on a patch to provide apr_[thread/proc]_mutex_timedlock() in APR, but did not finish the work mostly because of APR_ENOTIMPL on some mutex mechanisms (mainly Windows CRITICAL_SECTIONs which lack the functionality, making the APR abstraction quite useless IMO, or at least unix specific). Maybe I'll have a new look at it these days if I have the time to. Hmm, this remark about Windows only concerns thread-mutexes, proc-mutexes don't use a CRITICAL_SECTION but a HANDLE (which is then usable with WaitForSingleObject() that accepts a timeout). I think I'll give my patch a new chance, at least for proc-mutexes (and hence global-mutexes), but this is not a today's solution in any case since I would be at best an APR-1.6 feature (if accepted)...
Re: How to wait on a global lock with timeout
Hi Yann, On 30.09.2014 18:16, Yann Ylavic wrote: On Tue, Sep 30, 2014 at 5:30 PM, Yann Ylavic ylavic@gmail.com wrote: I have been working on a patch to provide apr_[thread/proc]_mutex_timedlock() in APR, [...] I think that is exactly what I was looking for... Your idea to use the native mutex functions is probably better than nothing. It provides the needed functionality at least for some platforms. I think I'll give my patch a new chance, at least for proc-mutexes (and hence global-mutexes), but this is not a today's solution in any case since I would be at best an APR-1.6 feature (if accepted)... That would be great. I am looking forward to test that, as soon as available... ;) Regards, Micha
Re: How to wait on a global lock with timeout
I join the line of those waiting for this feature in APR. I really would like to have it available -- Massimo Il 30/Set/2014 18:16 Yann Ylavic ylavic@gmail.com ha scritto: On Tue, Sep 30, 2014 at 5:30 PM, Yann Ylavic ylavic@gmail.com wrote: I have been working on a patch to provide apr_[thread/proc]_mutex_timedlock() in APR, but did not finish the work mostly because of APR_ENOTIMPL on some mutex mechanisms (mainly Windows CRITICAL_SECTIONs which lack the functionality, making the APR abstraction quite useless IMO, or at least unix specific). Maybe I'll have a new look at it these days if I have the time to. Hmm, this remark about Windows only concerns thread-mutexes, proc-mutexes don't use a CRITICAL_SECTION but a HANDLE (which is then usable with WaitForSingleObject() that accepts a timeout). I think I'll give my patch a new chance, at least for proc-mutexes (and hence global-mutexes), but this is not a today's solution in any case since I would be at best an APR-1.6 feature (if accepted)...
mod_proxy's aside connections proposal
Hello,
I have proposed a patch for PR39673 but I'm not sure it would be
accepted for mainline httpd, so here I am.
The patch adds the possibility (for a module) to acquire a connection
aside from the worker's reslist, so that it won't be acquired from the
reslist nor put back to it once released (nor reused for another
client connection, unless the module does it on its own).
The connection is still bound to a worker, hence it will be
established/closed according to the worker's configuration (hostname,
port, SSL, is_address_reusable, TTL...), and the module can still (and
should) call ap_proxy_determine_connection(),
ap_proxy_connect_backend() and ap_proxy_release_connection() to do
that work.
The new function to acquire an aside connection is :
PROXY_DECLARE(int) ap_proxy_acquire_connection_ex(const char *proxy_function,
proxy_conn_rec **conn,
proxy_worker *worker,
server_rec *s, void *baton,
ap_proxy_acquire_fn acquire,
ap_proxy_release_fn release);
When called with both baton and acquire set to NULL,
ap_proxy_acquire_connection_ex() is the same as
ap_proxy_acquire_connection(), and will acquire connections from the
reslist.
When acquire is not NULL, it is used as the constructor for *conn
(given the baton), so it's up to the caller to create a new (aside)
connection (with the new function ap_proxy_aside_connection() also
provided) or reuse an existing one according to its criteria (and the
baton). If release is not NULL, it will be called upon
ap_proxy_release_connection() with the same baton.
When acquire is NULL but baton isn't, a built-in acquire function is
used to select an existing or create a new connection associated to a
conn_rec (and still the worker). The baton is assumed to be a conn_rec
(eg. the one of the client's connection), and this mode is hence a
return back to the days of httpd = 2.0.
This is the trick to respond to PR39673 (NTLM forwarding), or any
issue due to a session associated to a TCP connection (some
load-balancers configured to do so expect that), which mod_proxy can't
forward currently.
The patch then uses the new ap_proxy_acquire_connection_ex() function
(latter mode) in mod_proxy_http and mod_proxy_ajp (which both have the
ability to reuse connections), but only when the environment var
proxy-aside-c is defined. This allows for the administrator to use
SetEnv[If] or a RewriteRule to enable the functionality, according to
specific conditions, like (PR39673) :
RewriteCond %{HTTP:Authorization} ^NTLM
RewriteRule ^ - [E=proxy-aside-c]
The patch is attached, and I tried to make it as generic as I could so
that it is not PR39673 (NTLM) specific, aside connections looks like a
nice to have for modules.
This version is slightly different from the one proposed in the PR, in
that it will always close aside connections associated with a
non-reusable worker upon release (the PR's one failed to do that), and
the default release callback (when none is given) for aside
connections is now to apply the worker's configured TTL (if any).
Do you think this can/should (not) be applied to httpd?
Regards,
Yann.
Index: modules/proxy/mod_proxy_ajp.c
===
--- modules/proxy/mod_proxy_ajp.c (revision 1628385)
+++ modules/proxy/mod_proxy_ajp.c (working copy)
@@ -730,6 +730,12 @@ static int proxy_ajp_handler(request_rec *r, proxy
ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(00895) serving URL %s, url);
/* create space for state information */
+if (apr_table_get(r-subprocess_env, proxy-aside-c)) {
+status = ap_proxy_acquire_connection_ex(scheme, backend, worker,
+r-server, r-connection,
+NULL, NULL);
+}
+else
status = ap_proxy_acquire_connection(scheme, backend, worker,
r-server);
if (status != OK) {
Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c (revision 1628385)
+++ modules/proxy/proxy_util.c (working copy)
@@ -72,6 +72,13 @@ static struct wstat {
{0x0, '\0', NULL}
};
+/* Opaque aside connection record */
+struct ap_proxy_aside_rec {
+void *baton;
+ap_proxy_release_fn release;
+apr_time_t expiry; /* idle connection expiry */
+};
+
/* Global balancer counter */
int PROXY_DECLARE_DATA proxy_lb_workers = 0;
static int lb_workers_limit = 0;
@@ -1351,6 +1358,14 @@ PROXY_DECLARE(int) ap_proxy_connection_reusable(pr
return ! (conn-close || !worker-s-is_address_reusable || worker-s-disablereuse);
}
+static void socket_cleanup(proxy_conn_rec *conn)
+{
+conn-sock = NULL;
+
Re: mod_proxy's aside connections proposal
Hi, On 30.09.2014 16:47, Yann Ylavic wrote: Do you think this can/should (not) be applied to httpd? I would love to see this applied to httpd. Regards, Micha
