Re: Compiling httpd module for Windows
On Fri, May 22, 2015 at 3:15 PM, Paul Klinkenberg p...@ongevraagdadvies.nl
wrote:
Hi list members,
I created a simple httpd module in C, called mod_cfml. It is a port of a
Perl-based one, which was written by Jordan Michaels. I teamed up with him
to do some updates and do the port.
- original mod_cfml.PM:
https://github.com/utdream/mod_cfml/blob/master/perl/mod_cfml.pm
https://github.com/utdream/mod_cfml/blob/master/perl/mod_cfml.pm
- new mod_cfml.SO:
https://github.com/paulklinkenberg/mod_cfml/blob/develop/C/mod_cfml.c
https://github.com/paulklinkenberg/mod_cfml/blob/develop/C/mod_cfml.c
Reason I am writing to the list, is my inability to compile this module
for Windows. I am not a much experienced C coder, hope the code doesn't
show that though. I tried multiple options, and spent over 16 hours in
trying to get the module compiled on Windows. But to no avail. On other
platforms, I just use apxs, and it works like a charm.
Bluntly asked: would somebody on this list be willing to compile this code
for me/the project, for Apache 2.4 on Windows 32/64 bit? Or show me a
working way how to do it myself?
The project is 100% free opensource software, meant to help people, not to
make money :)
Thanks in advance, kind regards,
Paul Klinkenberg
apxs for Windows: http://svn.apache.org/viewvc/perl/apxs/trunk/
I prefer cmake. This should work reasonably if your httpd install looks
like the one installed with the cmake build: (probably stuff is in the
right place)
If you try this and it works, let me know; I could put the generic one
(i.e., not cfml) on Github with a license and readme with a pointer in
the script. I don't know if this is really enough code to have a license
for; my only concern is that I wouldn't want anyone to see this with some
other code and think that it is not free to use more widely than the code
they found it with.
- CMakeLists.txt --
PROJECT(MODS C)
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
SET(modnames cfml)
INCLUDE_DIRECTORIES(${CMAKE_INSTALL_PREFIX}/include)
FOREACH(shortname ${modnames})
SET(modbase mod_${shortname})
SET(modbasename ${modbase}.c)
ADD_LIBRARY(${modbase} SHARED ${modbasename})
SET_TARGET_PROPERTIES(${modbase} PROPERTIES SUFFIX .so)
TARGET_LINK_LIBRARIES(${modbase}
${CMAKE_INSTALL_PREFIX}/lib/libhttpd.lib
${CMAKE_INSTALL_PREFIX}/lib/libapr-1.lib
${CMAKE_INSTALL_PREFIX}/lib/libaprutil-1.lib)
INSTALL(TARGETS ${modbase} RUNTIME DESTINATION modules)
ENDFOREACH()
- end -
Put this CMakeLists.txt file in the directory with your source.
In MS Visual Studio command prompt:
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=\path\to\httpd\install -G NMake Makefiles
-DCMAKE_BUILD_TYPE=RelWithDebInfo \path\to\your\source
nmake nmake install
--/--
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: mod_fcgid: WIN32 compile-error and problem with slepp for 1 second
Hello Michael,
This is a mailing list for developers of plug-in modules for httpd.
us...@httpd.apache.org and d...@httpd.apache.org are better choices. (I
guess these two topics are code-related so dev@ is the best choice.)
But see below. (If further discussion is needed outside of the bug db,
please post to d...@httpd.apache.org.)
On 04/22/2015 02:26 AM, Koperek, Michael wrote:
Hi,
using the current mod_fcgid 2.3.9 sources, im not able to compile on windows
using msdev 6.0:
Compiling...
fcgid_conf.c
…\fcgid_conf.c(815) : error C2065: 'JOBOBJECT_EXTENDED_LIMIT_INFORMATION' :
undeclared identifier
…\fcgid_conf.c(815) : error C2146: syntax error : missing ';' before identifier
'job_info'
…\fcgid_conf.c(815) : error C2065: 'job_info' : undeclared identifier
…\fcgid_conf.c(815) : error C2059: syntax error : '{'
…\fcgid_conf.c(827) : error C2224: left of '.BasicLimitInformation' must have
struct/union type
…\fcgid_conf.c(828) : error C2065: 'JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE' :
undeclared identifier
…\fcgid_conf.c(830) : error C2065: 'JobObjectExtendedLimitInformation' :
undeclared identifier
Seems to be not compilable code. Any ideas what to do?
You need a newer SDK from Microsoft that defines that API. (Maybe a web
search will help?) It would be fair to open a bug against mod_fcgid
asking for some compatibility with ancient SDK bundled with your
compiler, but I'm not sure anyone would be able to work on that.
There is a sleep for 1 second in fcgid_bridge.c, function handle_request.
One second is for my application to high (user has to wait for each image to
load on a web side with multiple images 1 second).
I changed the code from
apr_sleep(apr_time_from_sec(1));
to
apr_sleep(APR_USEC_PER_SEC / 50); // 20 msecs
That works fine.
Is it possible to configure the sleep time?
Please comment on this bug to raise awareness that it affects more people:
https://bz.apache.org/bugzilla/show_bug.cgi?id=56308
Kind regards
Michael Koperek
e.bootis ag
e.bootis ag
Aktiengesellschaft mit Sitz in Essen
Amtsgericht Essen HRB 21782
Vorstand: Dr. Karl Langenstein, Ludger Langenstein, Tim Langenstein
Aufsichtsratsvorsitzender: Prof. Dr. Martin Užik
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten
haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht
gestattet.
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is strictly
forbidden.
Re: Best practice for handling synchronous signals SIGFPE etc
On 04/20/2015 03:50 PM, Mark Taylor wrote: I found that ./server/mpm_unix.c is registering a handler (sig_coredump) for SIGFPE, SIGSEGV, and other synchronous signals. I'd like to handle at least these two in my module, using my own handler. But then how do I determine if the the handler is called on a request thread or a server thread? And I'd like to default to still run the sig_coredump() function if it's signal is not in my module. Does anyone have experience with this? Regards, Mark A hook is provided for a module to perform some processing for coredump-ing signals, but the hook can't be used to prevent httpd from performing the usual sig_coredump work. mod_whatkilledus is a module that uses that particular hook and understands when a request or connection is active. There's info and a download link at http://emptyhammock.com/projects/httpd/diag/index.html
Re: apr_file_open
On Thu, Dec 25, 2014 at 2:08 AM, Alan Nilsson anils...@apple.com wrote: Do files opened with apr_file_open get closed as part of the pool clean up? alan yes -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: Apache module development on Mac OS X
On Tue, Jun 17, 2014 at 8:44 AM, Sindhi Sindhi sindhi@gmail.com wrote: Thankyou Nick. Actually I was looking at options to build the Apache module on systems where Apache server is not installed (and hence apxs is not available), so that the module can be built on such build machines also. I can do this on Windows and Linux, which provide the load-time linking options. When I created the Xcode project, I chose the below settings - 1) Framework and Library 2) C/C++ Library 3) Type = Dynamic 4) Mach-O Type = Dynamic Library Is there any alternate method to build Apache modules on Mac other that apxs since my target machines may not always have Apache server installed. Regards, Sindhi Any recommended method would be based on the files in the build directory in an httpd install, which apxs uses. See config_vars.mk, for example. Treat the httpd build support files as a prerequisite of your product build. On Tue, Jun 17, 2014 at 6:01 PM, Nick Kew n...@webthing.com wrote: On Tue, 17 Jun 2014 17:27:15 +0530 Sindhi Sindhi sindhi@gmail.com wrote: Any help will be highly appreciated. You're trying to build the module as a program. You need to build it as a shared object (think DLL, since you seem to be from a Windows background). Apache provides a tool to do that, with all the right paths and linkage. Check apxs in the docs. -- Nick Kew -- Born in Roswell... married an alien... http://emptyhammock.com/ http://edjective.org/
Re: tcp/ip sockets in apache module
On Fri, May 23, 2014 at 1:27 PM, Jeremy Thompson jer...@warehousesports.com wrote: I'm trying to write a module for apache. I've successfully compile in a test module that doesn't do a whole lot yet. I would like to be able to use tcp/ip sockets in the module to talk to another server. I added in some test code that should connect to my listening server and say hello whenever a request is made. I'm not getting any errors upon compiling the module with the sockets stuff added but its not actually doing anything. Basically its ignoring my connect and send code. Does anyone have a small working example of this or am I just way off base in using TCP/IP for external communication to a module. Thanks. ~Jeremy You need to know how to step through your code with a source level debugger so you can figure out these sorts of problems on your own. Do you know how to do that? Are you trying but it isn't stopping in your code? Is there some other issue with source level debugging? -- Born in Roswell... married an alien... http://emptyhammock.com/ http://edjective.org/
Re: tcp/ip sockets in apache module
On Fri, May 23, 2014 at 1:58 PM, Jeremy Thompson jer...@warehousesports.com wrote: I'Ll give that a shot. I'Ve used source level debugging in IDE's but it looks like I'Ll be using GDB for this one. Thanks. Have a look at http://httpd.apache.org/dev/debugging.html On Windows, you can start httpd normally and attach to the httpd child process from the Visual Studio IDE in order to debug. On 5/23/14 10:41 , Jeff Trawick traw...@gmail.com wrote: On Fri, May 23, 2014 at 1:27 PM, Jeremy Thompson jer...@warehousesports.com wrote: I'm trying to write a module for apache. I've successfully compile in a test module that doesn't do a whole lot yet. I would like to be able to use tcp/ip sockets in the module to talk to another server. I added in some test code that should connect to my listening server and say hello whenever a request is made. I'm not getting any errors upon compiling the module with the sockets stuff added but its not actually doing anything. Basically its ignoring my connect and send code. Does anyone have a small working example of this or am I just way off base in using TCP/IP for external communication to a module. Thanks. ~Jeremy You need to know how to step through your code with a source level debugger so you can figure out these sorts of problems on your own. Do you know how to do that? Are you trying but it isn't stopping in your code? Is there some other issue with source level debugging? -- Born in Roswell... married an alien... http://emptyhammock.com/ http://edjective.org/ -- Born in Roswell... married an alien... http://emptyhammock.com/ http://edjective.org/
Re: Apache 2.4.4 - undefined symbol: ap_log_rerror
On Tue, Apr 2, 2013 at 3:21 AM, Ian B porj...@yahoo.com.au wrote: ap_log_rerror is part of the core libraries, see: http://ci.apache.org/projects/**httpd/trunk/doxygen/group__** APACHE__CORE__LOG.htmlhttp://ci.apache.org/projects/httpd/trunk/doxygen/group__APACHE__CORE__LOG.html so I think you're going to have trouble making this work without a recompile. The trouble you'll have is that even if you make up some hacks to get it to load it will crash and burn (structure field offset and size changes, etc.) whenever moving from one stable release to another (1.3-2.0, 2.0-2.2, 2.2-2.4, etc.). It absolutely must be rebuilt from source. -Original Message- From: oh...@cox.net Sent: Tuesday, April 02, 2013 7:20 AM To: modules-dev@httpd.apache.org Subject: Apache 2.4.4 - undefined symbol: ap_log_rerror Hi, We use a 3rd party module that works fine with Apache 2.2.x, but when we try it with Apache 2.4.4, we get the Subject error: undefined symbol: ap_log_rerror From searching, this is due to changes in Apache 2.4, and what I've seen says that the module needs to be re-compiled for Apache 2.4. However, we don't have access to the source for the module (it's an Oracle module), and they are not saying when or if they'd modify the module for Apache 2.4.x, so I was wondering if there is any way to work around this. What I'm thinking/wondering if it'd be possible to LoadModule whatever module exports that ap_log_rerror (and probably other ap_ symbols), then maybe this would work? So, I was wondering if anyone tell me which shared library/.so that ap_log_rerror is exported from? Thanks, Jim -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: apr_memcache operation timeouts
On Tue, Mar 12, 2013 at 1:56 PM, Jeff Trawick traw...@gmail.com wrote:
On Mon, Mar 11, 2013 at 3:50 PM, Joshua Marantz jmara...@google.com wrote:
ping!
Please don't hesitate to push back and tell me if I can supply the patch
or
update in some easier-to-digest form. In particular, while I have
rigorously stress-tested this change using mod_pagespeed's unit test,
system-test, and load-test framework, I don't really understand what the
testing flow is for APR. I'd be happy to add unit-tests for that if
someone points me to a change-list or patch-file that does it properly.
-Josh
I'll try hard to work on this in the next couple of days. It would be great
to have fixes in APR-Util 1.5.x, which we hope to work on later this week.
Attached is a first pass at getting your patch to apr trunk. Can you
confirm that I didn't break your tests? (It passes the apr tests, but
that may not mean much.)
On Thu, Nov 1, 2012 at 8:04 AM, Joshua Marantz jmara...@google.com
wrote:
I have completed a solution to this problem, which can be a drop-in
update
for the existing apr_memcache.c. It is now checked in for my module as
http://code.google.com/p/modpagespeed/source/browse/trunk/src/third_party/aprutil/apr_memcache2.c
.
It differs from the solution in
https://issues.apache.org/bugzilla/show_bug.cgi?id=51065 in that:
- It doesn't require an API change; it but it enforces the 50ms
timeout that already exists for apr_multgetp for all operations.
- It works under my load test (which I found is not true of the patch
in 51065).
For my own purposes, I will be shipping my module with apr_memcache2 so
I
get the behavior I want regardless of what version of Apache is
installed.
But I'd like to propose my patch for apr_memcache.c. The patch is
attached, and I've also submitted it as an alternative patch to bug
51065.
If you agree with the strategy I used to solve this problem, then please
let me know if I can help with any changes required to get this into the
main distribution,
On Mon, Oct 22, 2012 at 5:21 PM, Joshua Marantz
jmara...@google.comwrote:
I've had some preliminary success with my own variant of apr_memcache.c
(creatively called apr_memcache2.c). Rather than setting the socket
timeout, I've been mimicing the timeout strategy I saw in
apr_memcache_multgetp, by adding a new helper method:
static apr_status_t wait_for_server_or_timeout(apr_pool_t* temp_pool,
apr_memcache2_conn_t*
conn) {
apr_pollset_t* pollset;
apr_status_t rv = apr_pollset_create(pollset, 1, temp_pool, 0);
if (rv == APR_SUCCESS) {
apr_pollfd_t pollfd;
pollfd.desc_type = APR_POLL_SOCKET;
pollfd.reqevents = APR_POLLIN;
pollfd.p = temp_pool;
pollfd.desc.s = conn-sock;
pollfd.client_data = NULL;
apr_pollset_add(pollset, pollfd);
apr_int32_t queries_recvd;
const apr_pollfd_t* activefds;
rv = apr_pollset_poll(pollset, MULT_GET_TIMEOUT,
queries_recvd,
activefds);
if (rv == APR_SUCCESS) {
assert(queries_recvd == 1);
assert(activefds-desc.s == conn-sock);
assert(activefds-client_data == NULL);
}
}
return rv;
}
And calling that before many of the existing calls to get_server_line
as:
rv = wait_for_server_or_timeout_no_pool(conn);
if (rv != APR_SUCCESS) {
ms_release_conn(ms, conn);
return rv;
}
This is just an experiment; I think I can streamline this by
pre-populating the pollfd structure as part of the apr_memcache_conn_t
(actually now apr_memcache2_conn_t).
I have two questions about this:
1. I noticed the version of apr_memcache.c that ships with Apache 2.4
is
somewhat different from the one that ships with Apache 2.2. In
particular
the 2.4 version cannot be compiled against the headers that come with a
2.2
distribution. Is there any downside to taking my hacked 2.2
apr_memcache.c
and running it in Apache 2.4? Or should I maintain two hacks?
2. This seems wasteful in terms of system calls. I am making an extra
call to poll, rather than relying on the socket timeout. The socket
timeout didn't work as well as this though. Does anyone have any
theories
as to why, or what could be done to the patch in
https://issues.apache.org/bugzilla/show_bug.cgi?id=51065 to work?
-Josh
On Fri, Oct 19, 2012 at 9:25 AM, Joshua Marantz
jmara...@google.comwrote:
Following up: I tried doing what I suggested above: patching that
change
into my own copy of apr_memcache.c It was first of all a bad idea to
pull
in only part of apr_memcache.c because that file changed slightly
between
2.2 and 2.4 and our module works in both.
I was successful making my own version
Re: apr_memcache operation timeouts
, I think the proposed patch that Jeff pointed to above is not really working (as he predicted). This test was done without SIGSTOPing the memcached; it would timeout under our load anyway and thereafter behave poorly. I'm going to follow up on that bugzilla entry, but for now I'm going to pursue my own complicated mechanism of timing out the calls from my side. -Josh On Thu, Oct 18, 2012 at 10:46 AM, Joshua Marantz jmara...@google.com wrote: Thanks Jeff, that is very helpful. We are considering a course of action and before doing any work toward this, I'd like to understand the pitfalls from people that understand Apache better than us. Here's our reality: we believe we need to incorporate memcached for mod_pagespeed http://modpagespeed.com to scale effectively for very large sites hosting providers. We are fairly close (we think) to releasing this functionality as beta. However, in such large sites, stuff goes wrong: machines crash, power failure, fiber cut, etc. When it does we want to fall back to serving partially unoptimized sites rather than hanging the servers. I understand the realities of backward-compatible APIs. My expectation is that this would take years to make it into an APR distribution we could depend on. We want to deploy this functionality in weeks. The workarounds we have tried backgrounding the apr_memcache calls in a thread and timing out in mainline are complex and even once they work 100% will be very unsatisfactory (resource leaks; Apache refusing to exit cleanly on 'apachectl stop') if this happens more than (say) once a month. Our plan is to copy the patched implementation of apr_memcache_server_connect and the static methods it calls into a new .c file we will link into our module, naming the new entry-point something else (apr_memcache_server_connect_with_timeout seems good). From a CS/SE perspective this is offensive and we admit it, but from a product quality perspective we believe this beats freezes and complicated/imperfect workarounds with threads. So I have two questions for the Apache community: 1. What are the practical problems with this approach? Note that in any case a new APR rev would require editing/ifdefing our code anyway, so I think immunity from APR updates such as this patch being applied is not a distinguishing drawback. 2. Is there an example of the correct solution to the technical problem Jeff highlighted: it is apparently missing a call to adjust the socket timeout and to discard the connection if the timeout is reached. That sounds like a pattern that might be found elsewhere in the Apache HTTPD code base. Thanks in advance for your help! -Josh On Wed, Oct 17, 2012 at 8:16 PM, Jeff Trawick traw...@gmail.com wrote: On Wed, Oct 17, 2012 at 3:36 PM, Joshua Marantz jmara...@google.com wrote: Is there a mechanism to time out individual operations? No, the socket connect timeout is hard-coded at 1 second and the socket I/O timeout is disabled. Bugzilla bug https://issues.apache.org/bugzilla/show_bug.cgi?id=51065 has a patch, though it is apparently missing a call to adjust the socket timeout and to discard the connection if the timeout is reached. More importantly, the API can only be changed in future APR 2.0; alternate, backwards-compatible API(s) could be added in future APR-Util 1.6. If memcached freezes, then it appears my calls to 'get' will block until memcached wakes up. Is there any way to set a timeout for that call? I can repro this in my unit tests by sending a SIGSTOP to memcached before doing a 'get'? Here are my observations: apr_memcache_multgetp seems to time out in bounded time if I SIGSTOP the memcached process. Yes! apr_memcache_getp seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_set seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_delete seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_stats seems to hang indefinitely if I SIGSTOP the memcached process. That last one really sucks as I am using that to print the status of all my cache shards to the log file if I detected a problem :( Why does apr_memcache_multgetp do what I want and not the others? Can I induce the others to have reasonable timeout behavior? When I SIGSTOP memcached this makes it hard to even restart Apache, at least with graceful-stop. On a related note, the apr_memcache documentation http://apr.apache.org/docs/apr-util/1.4/group___a_p_r___util___m_c.html is very thin. I'd be happy to augment it with my observations on its usage and the meaning of some of the arguments if that was desired. How would I go about that? Check out APR
Re: Forcing Apache to exit on startup
On Mon, Oct 22, 2012 at 4:05 PM, Sorin Manolache sor...@gmail.com wrote:
On 2012-10-22 21:29, Joshua Marantz wrote:
Hi,
Our module has multiple confirmation parameters. There is a case where if
you have option A and option B, then you must also specify option C,
otherwise Bad things can happen that are a lot easier to debug on startup
than they are after the server is running.
I know how to force 'apachectl' to exit with a nice error message if I
don't like the value of option A or option B. I can do that in the
function-pointer I provide in my option-table by returning a non-null
char*
message.
But in this case I want to exit nicely from the function I have registered
with ap_hook_child_init, having noticed that option A and B are set but
not
C. Is that possible?
By nicely, I mean that the user types:
% sudo /etc/init.d/apachectl restart
Error in pagespeed.conf: if you have option A and B specified, you must
specify a value for option C.
At that point either the server would not be running, or it would still be
in whatever state it was previously in.
Is this possible?
Currently our solution is to log an error and call abort(), and it's not
very nice!
exit(APEXIT_CHILDFATAL) is better than abort(), but if at all possible
you should handle this in the parent
Don't do it in child_init. I don't think the child can stop the server.
I think apachectl spawns apache with an appropriate -k argument (e.g.
apache2 -k stop) to start/stop/reload apache. However, I don't think that
the child has sufficient privileges to stop the parent by exec'ing apache2
-k stop. Nor can it send the apropriate signal to the parent.
But you can exit nicely in post_config, which is run (as root) after the
conf is read but before the children are spawned. It suffices to return a
non-ok code for apache to exit. I _think_, I'm not sure, that what you log
in post_config does not go on the console but goes to the server log. But
maybe there's a way to put the error message on the console too.
where the output goes depends on the phase of initialization
1st time post-config is called: ap_log_error goes nowhere
subsequent times post-config is called: ap_log_error goes to the error log
Some other modules handle this issue with an open_logs hook that runs
before core's open_logs hook. The module can then log an error that
goes to the console and return an error so that startup is aborted.
Example from server/mpm/prefork/prefork.c:
static void prefork_hooks(apr_pool_t *p)
{
/* The prefork open_logs phase must run before the core's, or stderr
* will be redirected to a file, and the messages won't print to the
* console.
*/
static const char *const aszSucc[] = {core.c, NULL};
...
ap_hook_open_logs(prefork_open_logs, NULL, aszSucc, APR_HOOK_MIDDLE);
Sorin
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: apr_memcache operation timeouts
On Wed, Oct 17, 2012 at 3:36 PM, Joshua Marantz jmara...@google.com wrote: Is there a mechanism to time out individual operations? No, the socket connect timeout is hard-coded at 1 second and the socket I/O timeout is disabled. Bugzilla bug https://issues.apache.org/bugzilla/show_bug.cgi?id=51065 has a patch, though it is apparently missing a call to adjust the socket timeout and to discard the connection if the timeout is reached. More importantly, the API can only be changed in future APR 2.0; alternate, backwards-compatible API(s) could be added in future APR-Util 1.6. If memcached freezes, then it appears my calls to 'get' will block until memcached wakes up. Is there any way to set a timeout for that call? I can repro this in my unit tests by sending a SIGSTOP to memcached before doing a 'get'? Here are my observations: apr_memcache_multgetp seems to time out in bounded time if I SIGSTOP the memcached process. Yes! apr_memcache_getp seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_set seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_delete seems to hang indefinitely if I SIGSTOP the memcached process. apr_memcache_stats seems to hang indefinitely if I SIGSTOP the memcached process. That last one really sucks as I am using that to print the status of all my cache shards to the log file if I detected a problem :( Why does apr_memcache_multgetp do what I want and not the others? Can I induce the others to have reasonable timeout behavior? When I SIGSTOP memcached this makes it hard to even restart Apache, at least with graceful-stop. On a related note, the apr_memcache documentationhttp://apr.apache.org/docs/apr-util/1.4/group___a_p_r___util___m_c.htmlis very thin. I'd be happy to augment it with my observations on its usage and the meaning of some of the arguments if that was desired. How would I go about that? Check out APR trunk from Subversion, adjust the doxygen docs in the include files, build (make dox) and inspect the results, submit a patch to d...@apr.apache.org. -Josh -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: Accessing environment variables set by other modules
On Mon, Oct 1, 2012 at 7:34 AM, Christoph Gröver gro...@sitepark.com wrote: Hello Daniel, Just a quick suggestion; Have you tried r-user ? Tak! Really a good suggestion. r-user is set if it's run in the fixup hook. I still would like to know if it's possible to access variables set by other modules, but for the current development it'll be sufficient. Sometimes envvars are set directly into subprocess_env (e.g., handling of SetEnv/SetEnvIf). IOW, subprocess_env is the primary representation. But the REMOTE_USER and HTTP request header variables are a representation of information stored elsewhere (r-user, r-headers_in), and that envvar representation is created just before running an external process. A module should always look at the primary representation, in this case r-user. With kind regards. Christoph Grøver -- -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: aprmemcache question
On Thu, Sep 27, 2012 at 10:58 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: On Thu, Sep 27, 2012 at 4:29 PM, Joshua Marantz jmara...@google.com wrote: Thanks Ben, That might be an interesting hack to try, although I wonder whether some of our friends running mod_pagespeed on FreeBSD might run into trouble with it. I did confirm that my prefork build has APR built with APR_HAS_THREADS, which for some reason I had earlier thought was not the case. It should work, provided you linked against libapr. The FreeBSD man page says this: If dlsym() is called with the special handle NULL, it is interpreted as a reference to the executable or shared object from which the call is being made. Thus a shared object can reference its own symbols. And that's how it works on Linux, Solaris, NetBSD and probably OpenBSD as well. Do you have a feel for the exact meaning of that TTL parameter to apr_memcache_server_create? You mean what units it uses? Microseconds (at least, in 2.4). Right. I screwed up on changing that yesterday. The APR doc was already fixed long ago to indicate it was microseconds instead of seconds, the Subversion code hasn't been fixed to respect that, and the bug that was opened to fix the code to use seconds put me in the wrong frame of mind :( What does ttl mean for this particular API? All resources in the resource list are cleaned up when the memcache server is deleted/pool is cleared/destroyed. Individual resources are returned to the list at the end of individual memcache operations. When a resource is returned to the list, old resources are destroyed, where old is determined by the ttl. Destroying a memcache resource means it sends the quit message to memcached and closes the socket. So ttl sets a limit on how long a particular connection to memcached can be used. -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: aprmemcache question
On Thu, Sep 27, 2012 at 11:15 AM, Joshua Marantz jmara...@google.com wrote: On Thu, Sep 27, 2012 at 10:58 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: If dlsym() is called with the special handle NULL, it is interpreted as a reference to the executable or shared object from which the call is being made. Thus a shared object can reference its own symbols. And that's how it works on Linux, Solaris, NetBSD and probably OpenBSD as well. Cool, thanks. Do you have a feel for the exact meaning of that TTL parameter to apr_memcache_server_create? You mean what units it uses? Microseconds (at least, in 2.4). Actually what I meant was what that value is used for in the library. The phrase time to live of client connection confuses me. Does it really mean the maximum number of seconds apr_memcache is willing to wait for a single operation? Or does it mean *both*, implying that a fresh TCP/IP connection is made for every new operation, but will stay alive for only a certain number of seconds. TCP/IP connections, once created, will be retained for the specified (ttl) number of seconds. They'll be created when needed. The socket connect timeout is hard-coded to 1 second, and there's no timeout for I/O. It is a little disturbing from a module-developer perspective to have the meaning of that parameter change by a factor of 1M between versions. Would it be better to revert the recent change and instead change the doc to match the current behavior? The doc was already changed to match the behavior, but I missed that. The caller I know of used the wrong unit, and I'll submit a patch to fix that in the caller, as well as revert my screw-up from yesterday. -Josh -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: aprmemcache question
On Wed, Sep 26, 2012 at 5:38 PM, Joshua Marantz jmara...@google.com wrote: Hi, I've been having some success with the apr_memcache_* functions. In load-tests, however, I'm finding a lot of timeouts with apr_memcache_multgetp. Specifically, the status returned with the individual elements is APR_TIMEUP. This leads me to wonder what the significance of the second to last arg to this function is: apr_memcache_server_create( pool_, hosts_[i].c_str(), ports_[i], kDefaultServerMin, kDefaultServerSmax, thread_limit_, kDefaultServerTtlUs, server); I have kDefaultServerSmax initialized to 600, as that's the value I found in mod_socache_memcache.c But that seems stingy (if it's really in microseconds). Should I be giving that a few hundred millis instead? http://apr.apache.org/docs/apr-util/1.4/group___a_p_r___util___m_c.html#ga18ddd72bc1ab5edb0a08a8f26f193bd3 claims that means time to live of client connection but I don't understand what that phrase means exactly, or if it relates to the APR_TIMEUP returns I've been suffering from. My code is here; http://code.google.com/p/modpagespeed/source/browse/trunk/src/net/instaweb/apache/apr_mem_cache.cc -Josh d...@apr.apache.org is a better place to ask about details of apr functions. Coincidentally, earlier today I committed someone's fix for the confusion about the units of ttl: http://svn.apache.org/viewvc?view=revisionrevision=1390530 It is supposed to be in seconds. Pick up the tiny change to apr_memcache.c and see if that helps anything. You should continue this discussion on d...@apr.apache.org. -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: aprmemcache question
On Wed, Sep 26, 2012 at 7:31 PM, Joshua Marantz jmara...@google.com wrote: Looking at source, I see that Jeff's patch, and the 'ttl' parameter in general, is only referenced under '#if APR_HAS_THREADS'. When I load-tested and found the timeouts, I was testing under Apache 2.2 Prefork, and thus that patched code is not even compiled, IIUC. However I would still like to know what that parameter is for when running under Worker. I think the implication of my source journey is also that if my module instantiates multiple threads under Prefork (which it does), it must call apr_memcache* routines from only one of them. Is that correct? APR is usually compiled with thread support even when using the prefork MPM. Check APR_HAS_THREADS in apr.h. If APR_HAS_THREADS is 0 (very unlikely), you probably want to fail the build of your module to avoid having to worry about this. As far as what the parameter means... I'll try to look tomorrow if nobody replies first. See apu_version() for the APR-util version. -Josh On Wed, Sep 26, 2012 at 7:17 PM, Joshua Marantz jmara...@google.com wrote: +dev (sorry for the duplicate; my first attempt failed due to not being a subscriber). Keeping modules-dev on CC if that's appropriate. Thanks, Jeff, I was wondering if there was a units issue there. I'm still wondering if anyone can describe the meaning of that argument in more detail. Is that related to my multiget APR_TIMEUP returns? The phrase time to live of client connection confuses me. Would it be inaccurate to instead say the maximum number of seconds apr_memcache is willing to wait for a single operation? Or does it mean *both*, implying that a fresh TCP/IP connection is made for every new operation, but will stay alive for only a certain number of seconds. I have a practical question about how I release software given this change. Our module (mod_pagespeed) is documented to run with Apache 2.2 and Apache 2.4. It seems like for 2.2 I should probably multiple my desired argument by a million. Same with for 2.4.x and earlier, for some value of x. How should I work this in my code? Should I query the version number using an apr utility or something and multiply by a million in certain cases? What's the best practice calling this function for module developers? -Josh On Wed, Sep 26, 2012 at 6:20 PM, Jeff Trawick traw...@gmail.com wrote: On Wed, Sep 26, 2012 at 5:38 PM, Joshua Marantz jmara...@google.com wrote: Hi, I've been having some success with the apr_memcache_* functions. In load-tests, however, I'm finding a lot of timeouts with apr_memcache_multgetp. Specifically, the status returned with the individual elements is APR_TIMEUP. This leads me to wonder what the significance of the second to last arg to this function is: apr_memcache_server_create( pool_, hosts_[i].c_str(), ports_[i], kDefaultServerMin, kDefaultServerSmax, thread_limit_, kDefaultServerTtlUs, server); I have kDefaultServerSmax initialized to 600, as that's the value I found in mod_socache_memcache.c But that seems stingy (if it's really in microseconds). Should I be giving that a few hundred millis instead? http://apr.apache.org/docs/apr-util/1.4/group___a_p_r___util___m_c.html#ga18ddd72bc1ab5edb0a08a8f26f193bd3 claims that means time to live of client connection but I don't understand what that phrase means exactly, or if it relates to the APR_TIMEUP returns I've been suffering from. My code is here; http://code.google.com/p/modpagespeed/source/browse/trunk/src/net/instaweb/apache/apr_mem_cache.cc -Josh d...@apr.apache.org is a better place to ask about details of apr functions. Coincidentally, earlier today I committed someone's fix for the confusion about the units of ttl: http://svn.apache.org/viewvc?view=revisionrevision=1390530 It is supposed to be in seconds. Pick up the tiny change to apr_memcache.c and see if that helps anything. You should continue this discussion on d...@apr.apache.org. -- Born in Roswell... married an alien... http://emptyhammock.com/ -- Born in Roswell... married an alien... http://emptyhammock.com/
