Shared memory - Data between threads
I used a mutex in implementing mod_spamhaus, mod_sorbscheck and mod_torcheck to maintain consistency across threads. I have not seen problems since. I found the Apache mutex code to be somewhat opaque, so I wrote my own. I did find that "interesting" but after several revisions I had something I could return to later on and still understand, and (possibly more importantly) I felt it would be fairly clear to an unsophisticated reader. http://www.miim.com/software/linux/index.shtml
Module portability
I am not sure if this is a question for this list or for the APR forum, but it seems to me that this forum is more likely to have an answer. To what extent can I expect a module compiled on system A to be portable to system B: -- if both are running the same distribution of linux, though possibly not the same release -- if system A and system B are both running Apache 2.4.x, though possibly not the same release -- if system A has a custom-built Apache and system B has a distribution release of Apache I'm asking because several people don't want to compile their own modules (can't imagine why, it's so easy an Einstein can do it) and asked for ready-to-go kits that they can drop in and go. APR seems to me to be a black box, and if one of my modules calls a reslib that may not be present in another machine or at a different level, it seems to me there's potential problems there; and yet APR being a black box, it is going to be difficult to force static linking against libraries instead of resident libraries if APR intends to go that way.
Re: Chasing a segfault, part II
After two days of testing I can testify that Sorin's analysis is correct.
Sometimes Apache points to a user-agent string, sometimes it points to a null
string, and sometimes there is a null pointer.
The reason nothing appeared in the logs is because the requests coming in with
empty user-agent headers were going to the default server, not to one of the
named vhosts. The default server returns 410 on everything and logs nothing.
The test module should not have been active on the default server, but it is a
good thing it was; this would have been wretched if it had gone to the field
with that bug.
For future reference to anyone dealing with the user-agent string, here is the
code I adopted which handles all three possibilities. It is streamlined from
the original version and no longer copies the user-agent string to local
storage. This allows the CRC and DJB2 routines to process the entire string and
not a length-limited substring.
size_t ualength;
const unsigned char* uastring;
...
...
/* Retrieve the user-agent string */
uastring = apr_table_get(r->headers_in, "User-Agent");
/* If there is no user-agent string, the CRC/DJB2 defaults to x/x0 */
ua_crc = 0x;
ua_djb = 0x0;
/* If the user-agent string is empty, the CRC/DJB2 defaults to x0/x. */
if (uastring != NULL) {
ua_crc = 0x0;
ua_djb = 0x;
ualength = strlen(uastring);
/* If user-agent string exists, compute the CRC-32 and DJB2 hash */
if (ualength != 0) {
ua_crc = bc_crc_32(r, bc_scfg, uastring, ualength);
ua_djb = bc_djb2hash(r, bc_scfg, uastring, ualength);
}
}
My thanks to everyone who commented on the problem. I would not have found the
issue without assistance.
On Tuesday, 26 October 2021, 07:59:04 am GMT+1, Sorin Manolache
wrote:
On 26/10/2021 08.18, miim wrote:
> ua_pointer = apr_table_get(r->headers_in, "User-Agent");
> /* Find out how long the Apache-supplied string is */
> ualength = strlen(ua_pointer);
If the request does not contain any user-agent then ua_pointer will be
NULL. strlen of NULL will segfault.
S
Chasing a segfault, part II
My thanks to everyone for their input on this problem. While I was unable to
get the backtrace and whatkilledus modules to report on failure, I isolated the
cause to the following code in the handler.
The code itself does not segfault and indeed it appears to execute properly,
retrieving the user-agent string with correct length and logging it. However,
when this code is included Apache segfaults some time later. (It's not the
logging causing it; the segfault still occurs without the logging.)
I can not see why this code should be overwriting Apache data structures in
such a way as to cause Apache to segfault. I have rewritten it several
different ways and it still causes segfaults.
Might anyone have insight into this issue?
== HANDLER CODE ==
static int bridcheck_handler
(request_rec *r) {
const char *ua_pointer;
char useragent[UA_BUFFERSIZE];
size_t ualength;
size_t ualen2;
/* Retrieve the user-agent string */
/* Null the last byte in our buffer so that strings are always terminated
*/
useragent[UA_BUFFERSIZE-1] = '\0';
/* Load pointer to the Apache request record user-agent header field */
ua_pointer = apr_table_get(r->headers_in, "User-Agent");
/* Find out how long the Apache-supplied string is */
ualength = strlen(ua_pointer);
/* Copy only if there's something to copy */
if (ualength != 0)
/* Our buffer gets Apache's request record user-agent field */
/* Protect from segfault by limiting length at buffersize -1 */
strncpy(useragent, ua_pointer, UA_BUFFERSIZE-1);
/* Don't use the original strncpy below. Dissected the functionality
into pieces above. */
/* strncpy(useragent, apr_table_get(r->headers_in, "User-Agent"),
UA_BUFFERSIZE-1); */
/* Now that we have our prize ... how long is it? */
ualen2 = strlen(useragent);
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
"bc: ualength = %u, <%s>",
ualength, apr_table_get(r->headers_in, "User-Agent"));
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
"ualen2 = %u, <%s>",
ualen2, useragent);
return DECLINED;
}
== ERROR LOG EXTRACT
Oct 26 05:10:39 157.55.39.67 bc: ualength = 71, http://www.bing.com/bingbot.htm)>
Oct 26 05:10:39 157.55.39.67 ualen2 = 71, http://www.bing.com/bingbot.htm)>
Oct 26 05:11:11 71.6.232.7 bc: ualength = 115,
Oct 26 05:11:11 71.6.232.7 ualen2 = 115,
Oct 26 05:17:33 110.70.47.92 bc: ualength = 120,
Oct 26 05:17:33 110.70.47.92 ualen2 = 120,
Chasing a segfault
I have a relatively simple module which is nonetheless causing Apache to intermittently segfault. I've added debugging trace messages to be sent to the error log, but the lack of anything in the log at the time of the segfault leads me to think that the error log is not flushed when a message is sent. For example, a segfault occurs at 00:18:04, last previous request was at 00:15:36, so clearly the new request caused the segfault. But not even the "Here I am at the handler entry point" (see below) gets into the logfile before the server log reports a segfault taking down Apache. /* Retrieve the per-server configuration */ mod_bc_config *bc_scfg = ap_get_module_config(r->server->module_config, _module); if (bc_scfg->bc_logdebug & 0x00200) ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r, "mod_bridcheck: Enter bridcheck_handler"); I could turn on core dumping but (a) I am no expert at decoding core dumps and (b) I don't want to dump this problem on somebody else. So ... is there a way to force Apache to flush the error log before proceeding?
IPv4 vs IPv6 addressing
Sorin, thank you. I now have a small chunk of code that appears to do the job.
I do not have access to an IPv6 system to test with but it does identify the
connection type correctly on my IPv4 system.
I am not sure what APR_UNIX is, but it is referenced in the Apache source.
/* */
/* Testing code prefatory to including IPv6 support */
/* BEGINS */
/* */
switch(r->useragent_addr->family) {
case APR_INET6:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - IPv6", r->useragent_addr->family);
break;
case APR_INET:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - IPv4", r->useragent_addr->family);
break;
case APR_UNIX:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - Unix", r->useragent_addr->family);
break;
default:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - Unknown", r->useragent_addr->family);
break;
}
/* */
/* Testing code prefatory to including IPv6 support */
/*ENDS */
/* */
IPv4 vs IPv6 addressing
I've reviewed the last three years of the list and I can't find a commentary on this issue, nor was I able to find one on goofle. Consider an incoming request which might have either an IPv4 or an IPv6 address. The module wants to know which one. It is possible to sscanf the value in r->useragent_ip to see which format it matches. However, this is a relatively expensive operation for a small amount of info unless "most" are one or the other; then the test sequence can be optimized ... which according to Finagle's Law, will always be the wrong way around on somebody else's system. Is there a more efficient way to do this?
Re: Reference to core request fails during compile
Eric, you nailed the problem down precisely. Unfortunately while digging through the core source code it develops that getting the document root out of the core data structures is not a 100% reliable solution. Comments associated with the document root entries specifically state that If the URI has been internally rewritten to use a filename that is permitted but out of the server root, then the data in those entries does not apply to the specific request. This was all prompted by an issue in mod_nsf. The incoming URL was of the form http://www.server.com/index.html/wp-includes/wlwmanifest.xml but mod_nsf found r->filename to contain /www/server-root/index.html instead of (what I would consider to be correct) /www/server-root/index.html/wp-includes/wlwmanifest.xml which caused mod_nsf to say "Yes, the file's there, request is OK" instead of "Crafty, but still no such file, put 'em on the ban list." I'll need to think on this a bit more before going back to it.
Reference to core request fails during compile
I have undefined reference errors in an Apache module. I've cut the source
code down to a minimum that reproduces the error. Below is the source for
"mod_test.c" ...
#include "httpd.h"
#include "http_config.h"
#include "http_request.h"
#include "http_protocol.h"
#include "http_core.h"
#include "http_main.h"
#include "http_log.h"
#include "ap_mpm.h"
#include "apr_strings.h"
#include
#include
#include
#include
#include
#include
#include
module AP_MODULE_DECLARE_DATA test_module;
static int test_handler(request_rec *r);
static int test_init(apr_pool_t *p, apr_pool_t *plog, apr_pool_t *ptemp,
server_rec *s);
/* Structure containing state information for the module */
typedef struct {
} ns_mod_config;
static int ns_typematch(request_rec *r) {
ns_mod_config *ns_scfg = ap_get_module_config(r->server->module_config,
_module);
core_request_config *creq_cfg;
creq_cfg = ap_get_core_module_config(r->request_config);
return 0;
}
module AP_MODULE_DECLARE_DATA test_module = {
STANDARD20_MODULE_STUFF,NULL,NULL,NULL,NULL,NULL,NULL};
I am using a more-or-less standard Makefile for compiling the module (note that
the install option has been removed as this is a test to demonstrate the
problem.)
APXS=/usr/local/apache2/bin/apxs
APXS_OPTS=-Wc, -Wc,-DDST_CLASS=3
SRC=src/mod_test.c
OBJ=src/.libs/mod_test.so
$(OBJ): $(SRC)
@echo
$(APXS) $(APXS_OPTS) -c $(SRC)
@echo
@echo write '"make install"' to install module
@echo
clean:
rm -f src/.libs/*
rm -f src/*.o
rm -f src/*.lo
rm -f src/*.la
rm -f src/*.slo
rmdir src/.libs
The compile fails as follows:
/usr/local/apache2/bin/apxs -Wc, -Wc,-DDST_CLASS=3 -c src/mod_test.c
/usr/local/apache2/build/libtool --silent --mode=compile gcc -prefer-pic
-DLINUX -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -g -O2 -pthread
-I/usr/local/apache2/include -I/usr/local/apache2/include
-I/usr/local/apache2/include -DDST_CLASS=3 -c -o src/mod_test.lo
src/mod_test.c && touch src/mod_test.slo
src/mod_test.c: In function âns_typematchâ:
src/mod_test.c:34:3: error: unknown type name âcore_request_configâ
core_request_config *creq_cfg;
^~~
src/mod_test.c:35:14: warning: implicit declaration of function
âap_get_core_module_configâ [-Wimplicit-function-declaration]
creq_cfg = ap_get_core_module_config(r->request_config);
^
src/mod_test.c:35:12: warning: assignment makes pointer from integer without a
cast [-Wint-conversion]
creq_cfg = ap_get_core_module_config(r->request_config);
^
apxs:Error: Command failed with rc=65536
.
Makefile:23: recipe for target 'src/.libs/mod_test.so' failed
make: *** [src/.libs/mod_test.so] Error 1
I am not sure how this can occur. http_core.h is present in
/usr/local/apache2/include and it does include the definitions that are claimed
missing by the compile.
There is a nearly identical chunk of code used in Apache itself -- q.v.
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/server/protocol.c?view=markup#l110
Six other modules on the same system compile without errors, though none of
them use this specific reference to the core data structures.
I am forced to believe that it is a problem in the APXS compilation, though
there is so much going on behind the scenes that I wouldn't know where to start
looking.
