Re: Module portability
On 09/12/2021 02.58, miim wrote: I am not sure if this is a question for this list or for the APR forum, but it seems to me that this forum is more likely to have an answer. To what extent can I expect a module compiled on system A to be portable to system B: -- if both are running the same distribution of linux, though possibly not the same release -- if system A and system B are both running Apache 2.4.x, though possibly not the same release -- if system A has a custom-built Apache and system B has a distribution release of Apache I'm asking because several people don't want to compile their own modules (can't imagine why, it's so easy an Einstein can do it) and asked for ready-to-go kits that they can drop in and go. APR seems to me to be a black box, and if one of my modules calls a reslib that may not be present in another machine or at a different level, it seems to me there's potential problems there; and yet APR being a black box, it is going to be difficult to force static linking against libraries instead of resident libraries if APR intends to go that way. Linux distributions ship some 3rd-party modules (libapache2-mod-auth-kerb for kerberos authentication for example). I've noticed that there are regular updates of apache-2.4 packages without offering updates for the 3rd-party modules. So in many situations patch version number changes of apache-2.4 do not require rebuilds of the 3rd-party modules. The apache source API is guaranteed not to change among releases having the same major and minor version numbers. The immutability of the ABI (binary interface) does not really depend only on apache, but depends on your build tool chain too. For example if your code contained C++, some time ago there was a change in the ABI of C++ objects (std::string, std::list). (Debian/Ubuntu addressed this problem by creating packages with names such as libFOOv5 (note the v5). Slowly all libraries migrated to the new ABI and these names were phased out. So if your distributed your module as a package then it would not have installed because on one distribution it would depend on libFOO and on the other FOO is distributed in libFOOv5.) Another example: If your module depends on boost or on other header-only C++ library then your binary module will contain the code of the functions and the static data that it uses from boost. Let us assume that you've built it on distribution 1.0 with libboost 1.6 and your code uses an object of class X from libboost. On distribution 2.0 someone builds another module using boost 1.7 and uses the same object of class X. However class X has a different binary layout in boost 1.7 than in boost 1.6. So your module would function correctly as long as it is the only module using libboost that is loaded into apache. As soon as you have two modules using libboost, the runtime linker will load the object of class X symbol twice. It will retain only one instance. As the layout of the retained class X's symbol is incompatible with one of the modules that uses it, the execution will crash. There are solutions to such situations. You significantly increase the chances of binary compatibility if, when linking your module, you instruct the linker to put absolutely all symbols as hidden (private to the shared object) except the module structure. So the only global symbol of your binary module should be the module structure. (See ld's man page, the --version-script option.) Then the runtime linker, when loading the two shared objects of the two modules using boost, will load two instances of the class X object. They'll have different layouts, each used by its respective module and the crash is avoided. (Run "nm -aC mod_mymodule.so" to see all symbols of your module and consult nm's man page for an explanation of its output. Basically upper-case letters in the second column of the output denote global symbols and lower-case letter denote module-local symbols. Ideally the only globally defined symbol of your binary should look like this: "012192a0 D my_module". Note the upper-case D.) That said, it is very likely that the module build on one distribution is installable and running correctly on a more recent distribution. The situations in which it does not work are really idiosyncratic. I think you should not worry at all about APR. It is the least likely to cause a binary incompatibility between distributions. When the ABI of libapr changes it'll be big news. HTH, Sorin
Re: Chasing a segfault, part II
On 26/10/2021 08.18, miim wrote: ua_pointer = apr_table_get(r->headers_in, "User-Agent"); /* Find out how long the Apache-supplied string is */ ualength = strlen(ua_pointer); If the request does not contain any user-agent then ua_pointer will be NULL. strlen of NULL will segfault. S
Re: Chasing a segfault
On 23/10/2021 02.49, miim wrote:
I have a relatively simple module which is nonetheless causing Apache to
intermittently segfault.
I've added debugging trace messages to be sent to the error log, but the lack of anything
in the log at the time of the segfault leads me to think that the error log is not
flushed when a message is sent. For example, a segfault occurs at 00:18:04, last
previous request was at 00:15:36, so clearly the new request caused the segfault. But
not even the "Here I am at the handler entry point" (see below) gets into the
logfile before the server log reports a segfault taking down Apache.
/* Retrieve the per-server configuration */
mod_bc_config *bc_scfg = ap_get_module_config(r->server->module_config,
_module);
if (bc_scfg->bc_logdebug & 0x00200)
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
"mod_bridcheck: Enter bridcheck_handler");
I could turn on core dumping but (a) I am no expert at decoding core dumps and
(b) I don't want to dump this problem on somebody else.
So ... is there a way to force Apache to flush the error log before proceeding?
Hello,
I think it is not a problem of log flushing. It is just that when a
segfault occurs the death is sudden because the process is killed by the
OS and has few chances to handle the error itself.
I am very confident, almost 100% sure, that if you don't see the message
in the log then the execution has simply not reached it, the segfault
happened before.
In my opinion it is easier to learn some four or five gdb commands than
to do whatsoever when the segfault occurs. There's only one way of
preventing the death of the process and that it to place a handler on
the SIGSEGV signal in your module (see "man signal" or "man sigaction").
But there's not much you can do in the signal handler. As said, it is
much much easier to activate coredumps and learn some commands.
Here's how I do it typically:
In Debian/Ubuntu distributions, they put a file named envvars in
/etc/apache2. If you have such a distribution edit it as I show below.
If not, then make sure you get the same effects with other means.
I put the following two lines:
ulimit -c unlimited
echo 1 > /proc/sys/kernel/core_uses_pid
The first line is an internal shell command saying that there should be
no size limit on the core file. If you don't have /etc/apache2/envvars
then this command should be executed in the shell from which you launch
apache, such that the apache process inherits this configuration.
The second command instructs the kernel to add the process id to the
name of the core file. Thus, if you have two apache children that dump
cores at the same time, you'll get two different core files instead of
single file in which the kernel writes both cores, and makes it thus
unusable. If you don't have /etc/apache2/envvars then you can execute
this command in any shell, just that you need root privileges in order
to write to /proc/sys/kernel/core_uses_pid.
Let us assume you have now the core file and its name is core.12345,
where 12345 is the process id of the apache child process that died.
Then I start gdb and I execute the following gdb commands at the gdb prompt:
file /usr/sbin/apache2
core-file core.12345
thread apply all bt
The first command loads the apache executable.
The second command loads the core file.
The thirst command displays the call stacks of all threads of the
process (bt = backtrace).
You can switch between threads with the command
thread N
where N is the numerical id of the thread you want to switch to.
Once you're in a thread, you can move up and down the call stack with
the commands "up" and "down". If you compiled your module with debug
symbols then you can inspect variables with the "print" command, e.g.
"print bc_scfg". If, for example, the segfault occurred somewhere in a
libc function, such as malloc, free, strcpy, etc, you may move up the
call chain to the caller of the libc function, to inspect its arguments.
Besides the necessary "-g" compiler switch for adding debugging symbols,
I typically add the "-fno-inline -O0" switches. This prevents any code
optimisation. When I execute step-by-step in a debugger (a live program,
obviously, not a core-file) the instruction are really executed in the
order written in the program and not rearranged for speed.
You may also debug a live program. "Normal" programs, when debugging,
are typically launched directly in the debugger. This is not really
advisable in apache, because it forks. What I do is to let apache start
normally ("apache2ctl start" or "systemctl start apache2") and then
attach the debugger to a live apache child process. I launch gdb, then I
execute the following commands at the gdb prompt:
attach N (where N is the process id of the apache child)
break my_handler (set a breakpoint at one of my functions)
cont (let the process continue
Re: IPv4 vs IPv6 addressing
On 15/09/2021 02.22, miim wrote:
Sorin, thank you. I now have a small chunk of code that appears to do the job.
I do not have access to an IPv6 system to test with but it does identify the
connection type correctly on my IPv4 system.
I am not sure what APR_UNIX is, but it is referenced in the Apache source.
APR_UNIX denotes the family of Unix sockets. They appear in the file
system. They are not network sockets, i.e. a remote machine cannot
connect to a Unix socket. They are used like network sockets for
communication between processes on the same machine. man 7 unix.
I don't think that apache listens on Unix sockets, but I suppose it
could. Examples of applications that listen on Unix sockets are the
docker daemon and the X server.
/* */
/* Testing code prefatory to including IPv6 support */
/* BEGINS */
/* */
switch(r->useragent_addr->family) {
case APR_INET6:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - IPv6", r->useragent_addr->family);
break;
case APR_INET:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - IPv4", r->useragent_addr->family);
break;
case APR_UNIX:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - Unix", r->useragent_addr->family);
break;
default:
ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
" Family %d - Unknown", r->useragent_addr->family);
break;
}
/* */
/* Testing code prefatory to including IPv6 support */
/*ENDS */
/* */
Re: IPv4 vs IPv6 addressing
On 14/09/2021 04.58, miim wrote: I've reviewed the last three years of the list and I can't find a commentary on this issue, nor was I able to find one on goofle. Consider an incoming request which might have either an IPv4 or an IPv6 address. The module wants to know which one. It is possible to sscanf the value in r->useragent_ip to see which format it matches. However, this is a relatively expensive operation for a small amount of info unless "most" are one or the other; then the test sequence can be optimized ... which according to Finagle's Law, will always be the wrong way around on somebody else's system. Is there a more efficient way to do this? Hello, I've not tested this, but try inspecting r->connection->client_addr->family (or r->useragent_addr->family, it probably points to the same structure) If family is not initialized (though I think it is), then check ...->ipaddr_len or ...->salen. Have a look at /usr/include/apr-1.0/apr_network_io.h for all the fields of a apr_sockaddr_t structure. Regards, Sorin
Re: What is the best way of reading of post request body at hooks module from 2 hooks procedures (+) ?
On 17/05/2020 19.37, CpServiceSPb . wrote:
May you provide an example what is structure to store to ?
I tried to make some structure.
But it would become empty when running came to ap_hook_handler.
I suppose that I non correctly tried to get data at handler step saved
value at authn step from such structure.
Hello,
For example:
your_check_authn_callback(request_rec *r) {
/* read POST body as you explained */
MyStruct *obj = apr_pcalloc(r->pool, sizeof(MyStruct));
/* store what you extracted from the POST body in this obj */
obj->field1 = ...;
obj->field2 = ...;
...
/* store your obj in the request configuration */
ap_set_module_config(r->request_config, _module, obj);
}
your_handler_callback(request_rec *r) {
/* retrieve the object initialized and stored in the auth_callback */
MyStruct *obj = (MyStruct *)ap_get_module_config(r->request_config,
_module);
/* use obj->field1, obj->field2, etc according to your needs */
}
You don't have to parse everything in authn_callback, you could just
read the body, and store it unparsed:
your_check_authn_callback(request_rec *r) {
/* read POST body as you explained */
char *body = ...;
ap_set_module_config(r->request_config, _module, body);
/* extract user/pass/etc from body */
}
your_handler_callback(request_rec *r) {
/* retrieve the body from where you stored it: */
char *body = (char *)ap_get_module_config(r->request_config,
_module);
/* extract the other data from your body */
}
So the body (with ap_get_brigade) is read only once, namely in the first
callback, the check_authn callback. The second callback just retrieves
either the raw body from where you stored it or a structure that
contains data that were extracted from the body in the first callback.
HTH,
Sorin
вс, 17 мая 2020 г. в 10:32, Sorin Manolache :
On 15/05/2020 23.39, CpServiceSPb . wrote:
I write hook module for Apache2 consisted of hooks of 2 procedures:
ap_hook_check_authn/ap_hook_check_user_id and ap_hook_handler.
This module reads content of post method request, mainly body, and
extract
necessary information for functioning of the each of mentioned above
procedure.
For 1st one - it is user/password passed from clients at post request
body,
for 2nd one - it is some data.
That is reading post request body content from brigades/buckets and
saving
it into char buffer is called twice - from ap_hook_check_authn and and
ap_hook_handler.
But after callig reading post request body content reading function for
the
first time from ap_hook_check_authn, other calling of the function (from
ap_hook_handler) returns an empty body buffer.
Even other calling of post request body content reading function from
ap_hook_check returns already an emptybody biffer.
I read post body to a buffer at the following way:
1. apr_brigade_create
2. do-while cycle until eos_bucket is got set to 1
3. within the point 2 cycle ap_get_brigade
4. in for (bucket = APR_BRIGADE_FIRST(bb); bucket !=
APR_BRIGADE_SENTINEL(bb); bucket = APR_BUCKET_NEXT(bucket))
if bucket EOS then eos_bucket set to 1
else (transien bucket) apr_bucket_read is called and then all chunks are
reads to buffer
5. body buffer and its len is got finally
But in such pot body reading all buckest are deleted from brigade (not by
me) .
Hello,
I think you are doing everything fine.
You just cannot read the post body several times, because reading it is
consuming it.
So you just have to read it only once (in ap_hook_check_authn) and store
it yourself in a structure that belongs to your module. Then you can
reuse that structure in ap_hook_handler.
HTH,
Sorin
So what is the best way to read post body content to a buffer as many
times
as necessary as at ap_hook_check_authn step as at ap_hook_handler step
without loosing the content ?
Or may be it's more efficiency to read post body content once at
ap_hook_check_authn and then pass it to ap_hook_handler ?
But I don' t understand now how.
P. S.:
Way of calling of
r->kept_body = apr_brigade_create(r->pool, r->connection->bucket_alloc);
apr_bucket *bucketnew = apr_bucket_transient_create(bodycontent,
bodysize,
r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(r->kept_body, bucketnew);
after 1st calling of post request body content reading function worked
for
some versions ago (may be even for 2.2 only) .
But it doesn' t work now.
Re: What is the best way of reading of post request body at hooks module from 2 hooks procedures (+) ?
On 15/05/2020 23.39, CpServiceSPb . wrote: I write hook module for Apache2 consisted of hooks of 2 procedures: ap_hook_check_authn/ap_hook_check_user_id and ap_hook_handler. This module reads content of post method request, mainly body, and extract necessary information for functioning of the each of mentioned above procedure. For 1st one - it is user/password passed from clients at post request body, for 2nd one - it is some data. That is reading post request body content from brigades/buckets and saving it into char buffer is called twice - from ap_hook_check_authn and and ap_hook_handler. But after callig reading post request body content reading function for the first time from ap_hook_check_authn, other calling of the function (from ap_hook_handler) returns an empty body buffer. Even other calling of post request body content reading function from ap_hook_check returns already an emptybody biffer. I read post body to a buffer at the following way: 1. apr_brigade_create 2. do-while cycle until eos_bucket is got set to 1 3. within the point 2 cycle ap_get_brigade 4. in for (bucket = APR_BRIGADE_FIRST(bb); bucket != APR_BRIGADE_SENTINEL(bb); bucket = APR_BUCKET_NEXT(bucket)) if bucket EOS then eos_bucket set to 1 else (transien bucket) apr_bucket_read is called and then all chunks are reads to buffer 5. body buffer and its len is got finally But in such pot body reading all buckest are deleted from brigade (not by me) . Hello, I think you are doing everything fine. You just cannot read the post body several times, because reading it is consuming it. So you just have to read it only once (in ap_hook_check_authn) and store it yourself in a structure that belongs to your module. Then you can reuse that structure in ap_hook_handler. HTH, Sorin So what is the best way to read post body content to a buffer as many times as necessary as at ap_hook_check_authn step as at ap_hook_handler step without loosing the content ? Or may be it's more efficiency to read post body content once at ap_hook_check_authn and then pass it to ap_hook_handler ? But I don' t understand now how. P. S.: Way of calling of r->kept_body = apr_brigade_create(r->pool, r->connection->bucket_alloc); apr_bucket *bucketnew = apr_bucket_transient_create(bodycontent, bodysize, r->connection->bucket_alloc); APR_BRIGADE_INSERT_TAIL(r->kept_body, bucketnew); after 1st calling of post request body content reading function worked for some versions ago (may be even for 2.2 only) . But it doesn' t work now.
Re: Testing module without Apache
On 02/11/2019 11.00, Ervin Hegedüs wrote: Hi, this is just a theoretical question: is there any way to test/use an Apache module without Apache?n Hello, AFAIK no, but I didn't research it much. There are however best practices that try to come as close as possible to a "reasonable" degree of confidence in your code. You can separate your code in an application-dependent, apache-independent library on one hand and a generic, application-independent, apache-dependent glue code on the other hand and then you can test only the apache-independent library and "have faith" in your glue code. The apache-independent library must not refer to ap_* functions. So no ap_hook_*, ap_register_*_filter, ap_get|pass_brigade, ap_log_*, ap_rwrite, ap_rputs, ap_rprintf and many many other functions in your application-specific and apache-independent lib. Your application-specific lib may however use the data structures of apache (request_rec, server_rec, ap_filter_t) and of libapr (apr_tables, apr_pools etc). It's your responsibility however to initialize them in your test code, outside your lib. You cannot rely on ap_read_request for instance in order to initialize your request_rec structure. HTH, Sorin
Re: Is it safe to stop reading a bucket brigade in an input filter before the end?
On 16/05/2019 00.26, Paul Callahan wrote:
I have an apache body filter that copies off data from the incoming
request. But I terminate the reading of the data if the size of the
accumulated data is over some limit.
This is my function, it just breaks out of the loop when the max is read.
It seems to work ok. Do I need to do any additional cleanup or anything
because I did not go to the end?
Hello,
I suppose your code is ok.
Keep in mind that apache must clear the request body in order to be able
to parse and serve a new request that arrives on same, reused TCP
connection.
So towards the end of the processing of the current request, apache
calls a function, I think it's called ap_discard_request_body. This
function will read data from the network socket and pass it to the
filter chain. So your input filter will be called again (unless you
called ap_remove_input_filter somewhere), after you've already sent the
response to the client.
Best regards,
Sorin
Thank you
// returns 0 after desired length is read
int my_append_data(const char *data, apr_size_t len, void *request_ctx);
void my_body_read_done(void *request_ctx);
apr_status_t my_body_input_filter(ap_filter_t *f, apr_bucket_brigade *out_bb,
ap_input_mode_t mode,
apr_read_type_e block, apr_off_t nbytes,
void *request_ctx) {
request_rec *r = f->r;
conn_rec *c = r->connection;
apr_bucket_brigade *tmp_bb;
int ret;
tmp_bb = apr_brigade_create(r->pool, c->bucket_alloc);
if (APR_BRIGADE_EMPTY(tmp_bb)) {
ret = ap_get_brigade(f->next, tmp_bb, mode, block, nbytes);
if (mode == AP_MODE_EATCRLF || ret != APR_SUCCESS)
return ret;
}
while (!APR_BRIGADE_EMPTY(tmp_bb)) {
apr_bucket *in_bucket = APR_BRIGADE_FIRST(tmp_bb);
apr_bucket *out_bucket;
const char *data;
apr_size_t len;
if (APR_BUCKET_IS_EOS(in_bucket)) {
APR_BUCKET_REMOVE(in_bucket);
APR_BRIGADE_INSERT_TAIL(out_bb, in_bucket);
my_body_read_done(request_ctx);
break;
}
ret = apr_bucket_read(in_bucket, , , block);
if (ret != APR_SUCCESS) {
return ret;
}
// copy read data up to a limit of 1mb, then stop.
if (!my_append_data(data, len, request_ctx)) {
apr_bucket_delete(in_bucket);
break;
}
out_bucket = apr_bucket_heap_create(data, len, 0, c->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(out_bb, out_bucket);
apr_bucket_delete(in_bucket);
}
return APR_SUCCESS;
}
Re: request_rec.unparsed_uri missing scheme and host. parsed_uri missing most fields
On 14/05/2019 20.35, Paul Callahan wrote: Hello, I'm having trouble getting the full uri of a request from request_rec. The comment string for request_rec.unparsed_uri makes it sound like it should have the entire url, e.g. http:://hostname/path?etc. But it only has the path and the query parameters. The parsed_uri struct is populated with port, path and query paramters. Everything else (scheme, hostname, username, password, etc) is null. I set a breakpoint in "apr_uri_parse()" and verified the incoming *uri field only has the path and query parameters. Is this expected?How can I get the full URI? Hello, Yes, it is expected. When the client (meaning a program, not a human) makes a request it sends the following first line over the network connection: GET /path?arg1=val1=val2 HTTP/1.1 (I assume here that it uses the version 1.1 of the HTTP protocol.) In HTTP/1.1 a "Host" header must be present (it is not present in HTTP/1.0 but there is little HTTP/1.0 traffic nowadays) So you might get GET /path?arg1=val1=val2 HTTP/1.1 Host: www.example.com A browser will decompose the address http://www.example.com/path?arg1=val1=val2 that you type in its address bar and generate the two text lines shown above. But the server will not receive the string http://www.example.com/path?arg1=val1=val2 Moreover, http:// or https:// are not sent by the client. It's the server (apache) that determines (reconstructs) the scheme (i.e. http:// or https://) from the port and transport protocol (SSL/TLS or plain text) used by the request. The HTTP RFC (https://tools.ietf.org/html/rfc7230) has more details. Especially section 5.3 might be of interest to you. HTH, Sorin
Re: Modul command directive arguments
On 15/04/2019 22.39, Ervin Hegedüs wrote:
Hi,
I'm playing with a module, and found an interesting function.
Example:
const command_rec module_directives[] = {
...
AP_INIT_TAKE23 (
"DirectiveCmd",
cmd,
NULL,
CMD_SCOPE_ANY,
"Directive Command"
),
...
extern const command_rec module_directives[];
module AP_MODULE_DECLARE_DATA foo_module = {
STANDARD20_MODULE_STUFF,
create_directory_config,
merge_directory_configs,
NULL,
NULL,
module_directives,
register_hooks
};
And now if there is a command directive in the config, eg:
DirectiveCmd Arg1 "Arg2 re(foo)"
then I'll got the unescaped form of 2nd argument: "Arg2 re(\\foo)" (and of
course, it looks like all argument will unsescaped).
(It's new for me, because so far I've always used the "raw" stream reader
functions (eg. fread()) - nevermind :).)
Could anybody helps me please, which function parses the config file, and
make this unescaped formula (inside of Apache API)?
Hello,
The function that extracts words from a line of text is ap_getword_conf
declared in httpd.h.
It is called from ap_build_config declared in http_config.h.
The root of the call-chain is ap_read_config delared in http_config.h.
Best regards,
Sorin
Re: How to read data in a request handler and then return DECLINED without consuming the data in the bucket brigade?
On 2018-06-04 08:27, Paul Callahan wrote:
In apache modules, my understanding is if a handler declines a request, the
request is passed on to the next suitable handler. I'm finding though if
I read the bucket_brigade/request body, and then decline the request, the
subsequent handler doesn't get any data. It is like the act of reading the
bucket brigade consumes it.
I would like to have a request handler read the data, do some task (in this
case just count bytes), and decline the request without consuming the data
for the next handler.
Hello,
As far as I know, there is no simple way to do that.
Other handlers do something similar to what you've done, namely they
call ap_get_brigade(r->input_filters, ...).
So in order for them to still read something, you'll have to
* write an input filter
* in your handler you read the request body and store it somewhere
* afterwards in your handler you add your input filter to the chain of
input filters (ap_add_input_filter)
* your handler declines.
Then other handlers will call ap_get_brigade which will call your input
filter. Your input filter will not call any other filters but will copy
the stored body to the brigade passed in the call to your filter. Your
filter will give the illusion to other handlers that they are reading
from the network.
HTH,
Sorin
Thank you.
int my_declining_handler(request_rec *r) {
apr_status_t status;
int end = 0;
apr_size_t bytes, count = 0;
const char *buf;
apr_bucket *b;
apr_bucket_brigade *temp_brigade;
// here: header check for content-length/transfer encoding
temp_brigade = apr_brigade_create(r->pool,
r->connection->bucket_alloc);
do {
status = ap_get_brigade(r->input_filters, temp_brigade,
AP_MODE_READBYTES, APR_BLOCK_READ, BUFLEN);
if (status == APR_SUCCESS) {/* Loop over the contents of
temp_brigade */
for (b = APR_BRIGADE_FIRST(temp_brigade);
b != APR_BRIGADE_SENTINEL(temp_brigade);
b = APR_BUCKET_NEXT(b)) {
if (APR_BUCKET_IS_EOS(b)) {
end = 1;
break;
}
else if (APR_BUCKET_IS_METADATA(b)) {
continue;
}
bytes = BUFLEN;
status = apr_bucket_read(b, , ,
APR_BLOCK_READ);
count += bytes;
apr_bucket_delete(b);
}
}
apr_brigade_cleanup(temp_brigade);
} while (!end && (status == APR_SUCCESS));
if (status == APR_SUCCESS) {
return DECLINED;
} else {
return HTTP_INTERNAL_SERVER_ERROR;
}
}
Re: How to create ssl backend connections in a module?
On 2017-06-29 19:36, Christoph Rabel wrote: Hi, I have written an apache module that sometimes connects to a backend server. Currently it does that through http, open a socket, send a get request, get a response, process it. Nothing special. Now we need to support https too and I am wondering, how that could be accomplished. Should I use openssl directly? Does that work? Are there any helper functions I could use? I tried to find examples, but it is quite difficult since most of the examples cover configuration of ssl, not implementation of a ssl socket. I was also looking at mod_proxy but I don't understand how that stuff with the worker works. It's a lot of code and in the end I just need to open an ssl socket and I guess I can do the rest the same way as before. Any hints are appreciated. I should support Apache 2.2, but I might be able to weaken that to support only Apache 2.4, if that makes a huge difference. How do you do it now, in plain http? I see two or three ways in which you do it: using apache subrequests (ap_sub_req_method_uri), using mod_proxy (no code, just conf, like ProxyPass), using a 3rd-party library, such as libcurl or libneon for example. Or do you do it "manually", i.e. using the syscalls socket/connect/write, you write to the socket and implement the http protocol? The good news about the first three options is that they work with ssl without code modification. You just configure the URL of the backend and it recognizes https and performs the SSL handshake and communication. In my opinion (but it depends on your use case), the best option is mod_proxy. Check this generic way of configuring it: RewriteEngine On RewriteCond some_condition RewriteRule .* https://remote.host/path/to/remote/resource?args [P] https://remote.host/path/to/remote/resource> ProxyPass https://remote.host/path/to/remote/resource keepalive=On timeout=5 Your module processes requests to /your_url. If it has to make the request to the backend, then it sets some apache note or environment variable. The value of this variable is then checked in the RewriteCond. If the condition is satisfied then the request to /your_url is proxied to the remote.host backend. The response of the backend is then sent to your client. If you want to modify the response of the backend, or to send a completely different response to the client (and then you just use some data from the backend's response) then you write a filter and you activate it with the SetOutputFilter conf directive. This setup works with http and https. You just put the right scheme in the URLs in the conf. Hope this helps, Sorin Tia, Christoph
Re: modify request_rec->args
On 2016-03-25 00:59, Justin Kennedy wrote:
Hello,
I have a simple module, with just a quick_hander, it's sole function is to
check if there is a specific key=value on the query string, and modify the
value, so it gets picked up by a separate module.
For example: if "foo=1" is in r->args, then replace it with "foo=0",
decline the request so it gets picked up by the other module.
In my first attempt, I created a new string and assigned the pointer to
r->args, but it doesn't seem to "stick" when it gets to the second module.
Do I have to modify r->args directly, without changing the pointer? It's
been awhile since I've worked with C strings.
You don't need a module to do that. You can use some mod_rewrite
directives that you place inside your or :
RewriteEngine On
RewriteCond %{QUERY_STRING} ^(|.*&)foo=([^&]*)(&.*|$)
RewriteRule (.*) $1?%1foo=new_value%3
--
Sorin
Re: apr_shm_create succeeds then fails on Mac OS X
On 2015-12-25 19:36, Tapple Gao wrote: Hi. I’m trying to get mod_tile working on the builtin apache in Mac OS X El Capitan. I am running into a problem with apr_shm_create failing to allocate memory during ap_hook_post_config: [Fri Dec 25 12:09:17.898197 2015] [tile:error] [pid 22431] Successfully create shared memory segment size 888 on file /tmp/httpd_shm.22431 [Fri Dec 25 12:09:17.898285 2015] [tile:error] [pid 22431] (12)Cannot allocate memory: Failed to create shared memory segment size 2401448 on file /tmp/httpd_shm_delay.22431 Is there something I need to configure to get this shared memory working, or increase the limit? This module is most often run on Ubuntu linux, where it’s been running for years to power openstreetmap.org /* * Create a unique filename using our pid. This information is * stashed in the global variable so the children inherit it. * TODO get the location from the environment $TMPDIR or somesuch. */ shmfilename = apr_psprintf(pconf, "/tmp/httpd_shm.%ld", (long int)getpid()); shmfilename_delaypool = apr_psprintf(pconf, "/tmp/httpd_shm_delay.%ld", (long int)getpid()); I think that the location of the shmfile must be on a filesystem of a special type, namely tmpfs, which maps in memory and not on disk. Execute "mount" and check if you have such filesystems mounted. For example on my Linux machine: $ mount /dev/sda6 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered) tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=398160k) /dev/sda9 on /tmp type ext4 (rw,relatime,data=ordered) As you see, / and /tmp are of disk partitions while /run/shm has a filesystem of type tmpfs. I suggest to change the code and use a different location from /tmp/... On Linux the shared memory is often created in /run/shm/*. I have no experience with Mac OS X. The cleanest way would be to implement what's written in the commentary of the code above, namely the possibility to specify the path by an evrionment variable or from a configuration directive. Sorin
Re: Apache add content-length
On 2015-12-09 15:02, Pechoultres Xavier wrote: Hi everybody, I wrote an apache (2.2.x) module for special c++ app. It works fine for many years but a problem resist to me. Apache automatically add a content-length header even I send a Transfer-Encoding: chunked I set Transfer-Encoding: chunked in cgi module, I’m sure I do not set a content-length here. I unset Content-length in my module : apr_table_unset(r->headers_out, "Content-Length »); I’ve tried lot of stuff without success, and Apache always add a content-length. This create bug with recent WebCore application. If somebody can give me the light ! I've checked the sources of the apache server. Apache adds some filters to the request processing chain, two of them being ap_content_length_filter and ap_http_header_filter. ap_content_length_filter executes before ap_http_header_filter. ap_content_length_filter sets the Content-Length header if it can determine the size of the response body. ap_http_header_filter sets r->chunked only if the Content-Length header is not set. The response is chunked and the "Transfer-Encoding: chunked" header is set only if r->chunked = 1. In short: if apache can determine the size of the response message then you cannot force it to send "Transfer-Encoding: chunked" and it will always set the Content-Length header. -- Sorin
Re: Tracking sent responses
On 2015-11-06 15:12, Julien FROMENT wrote: Hello, We would like to use Apache HTTP Server to keep track of exactly what part of the response was sent over a socket. Apache could call an API asynchronously with the number of bytes sent over the socket for a given request. Here is the pseudo code: -- Client send a request -- Apache processes the request and send it to the backend server ... -- The backend server returns the response -- Apache sends the response to the client -- Apache calls Async API with the number of bytes sent Relaying the client request to a backend server may be realised with the ProxyPass and RewriteRule directives. Apache may log the number of bytes sent. See http://httpd.apache.org/docs/current/mod/mod_log_config.html#formats, the %B and %O flags. The log can be configured to be 1. appended to a file 2. sent to syslog (which in turn may forward it over udp/tcp to a log-host) 3. piped to an external program The first two options do not require any development from your part. If you really need that the number of bytes is sent to an http server, then you could write an external program that reads one line at a time from standard input and sends the line that it read to an http server. Regards, Sorin
Re: server directives are lost when activating module in vhost
On 2015-10-21 15:45, Justin Kennedy wrote: Greetings, I have these two directives specified in the root httpd.conf: ServerTokens Prod ServerSignature Off Those directives are being honored and all is well, until I activate my module within a virtual host. Once that happens, these directives are ignored. Is it possible for my module to be interfering with the other directives outside of my module configuration? If so, I'm thinking this could this be happening in my merge configuration hook, even though I only deal with directives related to my module. Any ideas? As far as I can imagine I cannot see how a module can interfere with the directives of another module (unless it handles the same directives). What happens if you disable your merge hook? Can you post your merge callback here? For debugging, it would be helpful if I could output the value of this directive in the various methods of my module. How can I access the value of this directive from within my module? These directives belong to the "core" module. So, in theory, you should core_conf_object = ap_get_module_config(conf_vector, _module); and then read the values from the core_conf_object. In practice however you can't do that because neither the address of core_module nor the definition of the core_conf structure are available to third-party modules. -- Sorin
Re: graceful child process exit
On 2015-09-21 00:45, Massimo Manghi wrote: Hi, I'm working on an issue with mod_rivet, a content generator module that embeds in Apache the interpreter of the Tcl scripting language. I won't bother you with the details of the problem (but if anyone is interested I'm ready to answer any questions) and let me put the question in short form: we need to give the module the ability to gently exit a child process, something like the function clean_child_exit I found in both prefork.c and worker.c MPMs that would do exactly what I need (delete the pChild pool and trigger the associated cleanup functions) but I could not find a public interface to it. Is there a public interface to achieve this functionality? The function ap_mpm_safe_kill at first looked a good candidate but I could not find documented if it's *the right way* Have a look at apr_pool_cleanup_register. I don't have pleasant memories with process pools. The problem is that sometimes apache children take a long time to exit. When this happens then the parent sends them signals in order to stop them, the signals becoming progressively stronger (first sigterm, then sigkill if I remember correctly). I do not remember the details, but I've been getting segfaults at process exit, so I'm steering clear of process pools since. Have a look if the conf pool is what you'd need. It is cleaned up every time the apache configuration is reloaded (the parent apache process stays alive, the apache worker children are stopped and a new generation of apache children is created). You can use the second callback of the apr_pool_cleanup_register to clean up things in the children. The difference between the conf pool and the process pool is the following: the process pool is passed to the post_config and child_init callbacks. The conf pool is not passed to child_init. So, in order to initialise things in the conf_pool you'll need to set a callback in the post_config hook. The post_config hook is executed as root in the parent process, before it forks a new generation of children, every time the apache configuration is reread (after each apache2ctl graceful). The child_init hook is executed in the apache child every time the child process is created and it executes without root privileges. Sorin
Re: Best practice for handling synchronous signals SIGFPE etc
On 2015-04-20 21:50, Mark Taylor wrote: I found that ./server/mpm_unix.c is registering a handler (sig_coredump) for SIGFPE, SIGSEGV, and other synchronous signals. I'd like to handle at least these two in my module, using my own handler. But then how do I determine if the the handler is called on a request thread or a server thread? And I'd like to default to still run the sig_coredump() function if it's signal is not in my module. Have a look at the man-page of sigaction and getcontext. When you set a signal handler you get the old signal handler (3rd argument of sigaction). So you can store it in a variable. In your own signal handler you do want you intend to do and at the end you call the old signal handler. In this way you can call sig_coredump. However you have to make sure that you set your signal handler _after_ apache has set his. Otherwise apache will replace yours. Have a look at the siginfo_t structure that is passed by the OS to your handler. You can get the program ID and the user ID from that structure. But not the thread apparently. Anyway, at least you can determine if the signal was raised in the parent or one of the worker children. Look also at the ucontext structure (man getcontext) that is passed to your signal handler. Maybe you can determine the source of the signal from that structure, though I think it's too close to machine code and registers to be useful. Alternately, you could use a thread-local variable (https://gcc.gnu.org/onlinedocs/gcc-3.3/gcc/Thread-Local.html). The first thing you do when you enter each function of your module is to set the variable. Whenever you exit a function you reset it. Thus, you may determine in your signal handler by inspection of the variable if the signal was raised by your module. (This works only if the signal handler is executed in the thread where the signal was raised which is not always the case. Otherwise you'll set some variable in your thread and read another one in the handler. Here's some information: http://stackoverflow.com/questions/11679568/signal-handling-with-multiple-threads-in-linux. Apparently the handlers for SIGSEGV and SIGFPE are called in the thread that raised them but it's not clear.) Sorin
Re: Mod_proxy is truncating Response Header size
On 2014-11-26 13:46, KPONVI Komlan (EXT) wrote: Hi everyone , I am having an issue with mod_proxy. I have to forward request to a server which add a header into the response. The size of that header is higher than 8k, and i notice that mod_proxy truncate that header before forward back the response to the initial caller. Try changing the value of this configuration directive: http://httpd.apache.org/docs/2.4/mod/core.html#limitrequestfieldsize Did somebody face that kind of problem? Cordialement, komlan KPONVI e-mail : komlan.kponvi-...@ca-technologies.fr mailto:komlan.kponvi-...@ca-technologies.fr Ce message et toutes les pieces jointes (ci-apres le message) sont etablis a l'intention exclusive de ses destinataires. Si vous recevez ce message par erreur, merci de le detruire et d'en avertir immediatement l'expediteur par e-mail. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. Les communications sur Internet n'etant pas securisees, l'expediteur informe qu'il ne peut accepter aucune responsabilite quant au contenu de ce message. This mail message and attachments (the message) are solely intended for the addressees. It is confidential in nature . If you receive this message in error, please delete it and immediately notify the sender by e-mail. Any use other than its intended purpose, dissemination or disclosure, either whole or partial, is prohibited except if formal approval is granted. As communication on the Internet is not secure, the sender does not accept responsibility for the content of this message.
Re: output filter needs to redirect to 503 error status
On 2014-10-16 15:36, Eric Johanson wrote: Hi, I have an output filter module which is working just fine, but I need to add a feature so that when certain error conditions occur during processing, the output filter hook function redirects the whole request to a 503 error status (service unavailable). Obviously for a handler module this is trivial to accomplish, but it is not clear how to do this in an output filter module. My output filter hooked function is defined as follows: apr_status_t mts_out_filter(ap_filter_t *f,apr_bucket_brigade *bb) I need this function to do something that causes the whole request to be redirected such that the client sees a 503 error status with no body/content. Things that I've tried so far: * Returning HTTP_SERVICE_UNAVAILABLE from the output filter function after calling ap_pass_brigade(f-next,bb) * Setting f-r-status to HTTP_SERVICE_UNAVAILABLE after calling ap_pass_brigade(f-next,bb) Try setting f-r-status _before_ calling ap_pass_brigade. If you get the 503 but you get the default 503 error response body that is automatically set by apache then replace it by using the ErrorDocument directive http://httpd.apache.org/docs/2.2/mod/core.html#errordocument (or http://httpd.apache.org/docs/2.4/mod/core.html#errordocument). Sorin * calling ap_send_error_response(f-r,HTTP_SERVICE_UNAVAILABLE) None of these really seem to behave properly. I just want the client to receive a 503 error status with no content body. There must be a way to achieve this behavior from within an output filter hook? Any advice is appreciated. Thanks, -Eric
Re: output filter needs to redirect to 503 error status
On 2014-10-16 22:35, Eric Johanson wrote: Thank you for the suggestion. I did try that, but the order in which you set f-r-status and call ap_pass_brigade doesn't seem to really make a difference. Basically what happens is that the browsers don't like the format of the HTTP response packet. They complain that there is extra unexpected data after the end of the response. Oddly, when I use the Linux wget command, it doesn't complain. I may just write a handler hook specifically for returning error codes. Then when the filter modules has an error, it can call ap_internal_redirect to a special page whose exclusive purpose is to be captured by my error handler to return an HTTP status code. Any other suggestions? This expected data message may be caused by a response body that has a different (bigger) length than what is announced in the Content-Length output header. If possible, try to not pass any data down the filter chain to the network when you want to set a 503 error. Set f-r-status = 503 and then pass a brigade that contains only an EOS bucket. I don't know if it is possible to send an empty body with a correct Content-Length (i.e. equal to zero) once your filter has already passed some data down the filter chain. I found some old code of mine that sends a 5xx code with an empty body when it detects an error. However my filter buffers all the data sent by a backend before it passes the filtered response in a single brigade down the filter chain. I.e. my filter returns APR_SUCCESS without invoking ap_pass_brigade whenever the brigade passed to my filter does not contain an EOS bucket. When the brigade contains an EOS bucket my filter passes the entire filtered response down the filter chain. So in my case, no data has reached any downstream filters before I detect an error. // nominal part ... // error-handling request_rec *r = f-r; r-status = HTTP_INTERNAL_SERVER_ERROR; ap_remove_output_filter(f); f-ctx = 0; r-eos_sent = 0; apr_table_t *tmp = r-headers_out; ap_add_output_filter(error_filter, 0, r, r-connection); r-headers_out = r-err_headers_out; r-err_headers_out = tmp; apr_table_clear(r-err_headers_out); return ap_pass_brigade(r-output_filters, bb); Apparently I remove the useful filter when I detect an error (ap_remove_output_filter(f)) and I replace it with another filter (ap_add_output_filter), that produces the error response. I suppose you don't have to do this, I suppose that the effect can be achieved in the same filter. I do not really remember why I swap the error and output headers. I suppose I clear the error headers in order to get rid of a previously computed Content-Length header. Also I do not remember why I reset the eos_sent flag, but I think this is important. And I'm quite surprised that I pass to the head of the output filter chain (ap_pass_brigade(r-output_filters, bb) and not f-next). I'm sorry that my explanations are incomplete, it's really an old code and I do not remember the details. But it's still in production and does what you want: it sends an empty-bodied response with a 5xx http status code. Sorin Thanks, -Eric -Original Message- From: Sorin Manolache [mailto:sor...@gmail.com] Sent: Thursday, October 16, 2014 12:59 PM To: modules-dev@httpd.apache.org Subject: Re: output filter needs to redirect to 503 error status On 2014-10-16 15:36, Eric Johanson wrote: Hi, I have an output filter module which is working just fine, but I need to add a feature so that when certain error conditions occur during processing, the output filter hook function redirects the whole request to a 503 error status (service unavailable). Obviously for a handler module this is trivial to accomplish, but it is not clear how to do this in an output filter module. My output filter hooked function is defined as follows: apr_status_t mts_out_filter(ap_filter_t *f,apr_bucket_brigade *bb) I need this function to do something that causes the whole request to be redirected such that the client sees a 503 error status with no body/content. Things that I've tried so far: * Returning HTTP_SERVICE_UNAVAILABLE from the output filter function after calling ap_pass_brigade(f-next,bb) * Setting f-r-status to HTTP_SERVICE_UNAVAILABLE after calling ap_pass_brigade(f-next,bb) Try setting f-r-status _before_ calling ap_pass_brigade. If you get the 503 but you get the default 503 error response body that is automatically set by apache then replace it by using the ErrorDocument directive http://httpd.apache.org/docs/2.2/mod/core.html#errordocument (or http://httpd.apache.org/docs/2.4/mod/core.html#errordocument). Sorin * calling ap_send_error_response(f-r,HTTP_SERVICE_UNAVAILABLE) None of these really seem to behave properly. I just want the client to receive a 503 error status with no content body. There must be a way to achieve
Re: Sharing information across Apache child processes
On 2014-09-29 13:39, Rajalakshmi Iyer wrote: Hello, I have a requirement whereby my application's configuration information (comprising a few complex data structures) needs to be shared across the various Apache child processes. Currently, the configuration is being individually loaded by each child process, which makes it hard for configuration changes to propagate. What is the best way / place to have a common configuration for the application? Please advise. I suppose you want to update the configuration without running apache2 -k graceful (or apache2ctl graceful). In this case you could use a segment of memory that is shared across the apache children. You'll have to create the shared segment memory before the parent forks its children (for example in post_config). The shared memory is then inherited by the forked children. You'll need a method to update the contents of the shared memory segment and a multiple-readers-single-writer inter-process exclusion mechanism in order to safely read and write from the shared segment. Sorin
Re: apr_hash_t and global scope
On 2013-12-11 19:17, Ingo Walz wrote: Hello modules-dev! I've encountered a problem with an apr_hash_t in the global scope of my module. Let me explain the use-case a bit. Every time, when an asset (image e.g.) is delivered via apache, it stores meta information of that file with a simple key/value pair in an apr_hash_t. That variable is part of the global scope of my module and allocated in register_hooks section (with the pool available here). If HTML is delivered that references an asset with meta information available, it rewrites the HTML based on the data from apr_hash_t. My problem is: using -X (only one worker) everything is fine and the module works as expected. But starting httpd with more than one worker, I sometime have no data on my apr_hash_t, which is expected to be there. I've tried various things, e.g. using server-process-pool and child_init to allocate the data, but the issue remains the same. I'm also using no global/thread_mutex, because I'm never reading and writing in the same thread (but in theory in the same process) - but I've no idea yet how hash_set is doing it internally, so this might be still a todo (is it? Do I really need a global mutex for data across worker/threads? Can I avoid it?). Using memcache is an option in theory, but I'd love to avoid it too. Otherwise my module scales much different, based on the delivered HTML. But anyways, it's most likely an issue with the wrong pool or with a misunderstanding of the scope and the cleanup of those pools. I'm making the assumption that you're using Unix and not Windows. I don't think it is related to pools or their cleanup. It is rather because of how Unix processes work. The request that updates your hash is served by a thread in one apache child and the request that reads from your hash may be served by a thread in another apache child. The problem is that each apache child has its own copy of the hash. What's the best way to use a everywhere shared apr_hash_t? Do I need apr_shm to work properly with the workers (or apr_rmm in my use case) together with a global_mutex or should I try to dig into socache and avoid doing this handling manually? Many outdated resources around the web (and even different internal implementations for the same use-case) made me feel a bit ... doubtful to come to a final decision. Maybe you can help me here?! :-) My advice would be to avoid doing it manually. Regards, Sorin Regards, Ingo
Re: response handling inside ap_hook_create_request cb function
On 2013-09-27 11:11, Pon Umapathy Kailash S wrote:
Thanks for your response, Sorin.
My concern in this approach is that - it would require one worker
thread to be held up for as long as this connection is open(and this
might be long + the number of clients can be higher than the worker
threads for my use-case/platform).
Given that the 1st handshake message is over http, i can setup the
connection in a handler hookup function and return a response, freeing
up the worker thread while keeping the connection persistent(plus save
the socket/ssl in a cache shared the worker threads).
Anyway in normal apache, i.e. over http, the worker threads are not
freed up after processing a request. An idle worker thread is assigned
to a connection when one is opened by a client for the whole duration of
the connection. You can check that in
modules/http/http_core.c:ap_process_http_sync_connection. You'll see
that ap_process_request is called in a loop. The loop is left when the
connection is closed. If the connection is idle, the worker thread is in
KeepAlive state. You can check this when looking at /server-status.
So it would not make any difference if you assigned a worker to the
connection in process_connection.
You'll have to take care though not to overuse the connection memory
pool because the pool is not destroyed while the connection is open, and
this could be a long time.
Now, when the next set of messages come in(which is not over http), I
would need to intercept these and add my handling(ideally write
something on the socket on which the message came and be done with the
request while keeping the connection persistent unless the message was
a control frame to close).
Regards,
Umapathy
On Fri, Sep 27, 2013 at 12:29 PM, Sorin Manolache sor...@gmail.com wrote:
On 2013-09-27 03:06, Pon Umapathy Kailash S wrote:
Hi,
Here is a quick background on what I am trying to do(basically adding
support for websockets - in a slightly customised manner as needed for
my app):
- Handle the initial handshake inside a cb function registered as a
handler hook(from here, I compute checksums required and return the
response headers as needed).
Also, the socket from which the request was read is stored in a cache.
- For subsequent message reception(on the same connection), i have a
function cb registered using ap_hook_create_request(since this is a
different protocol format message). Here, I read and parse the
messages/requests which are coming in from the cached list of
sockets(this is working).
However, once I return from this cb, the connection/socket seems to be
closed. I guess the request is further passed down to hooks down the
line and the connection is closed since the req format is not known.
What would be the best way to handle this scenario?
I have the following in mind:
- let the request not be processed any further(and keep the connection
on).
- create a req structure with dummy http headers that i can later
recognise and handle inside my handler hook to just ignore later on
are there any examples/notes on how these can be achieved?
In my opinion, it is too late to handle non-http in the create_request
callback.
The create_request callback is called from
ap_run_process_connection-ap_process_http_{sync|async}_connection-ap_read_request.
Your create_request callback returns to ap_read_request from where the
request is further processed as an http request.
In my opinion you should short-cut the http processing and hook
ap_hook_process_connection. However, there, in process_connection, you have
no request_rec, you have just a conn_rec. process_connection is called only
once per connection creation. So it should handle all the requests that
arrive while the connection is open.
Sorin
Regards,
Umapathy
Re: How to determine the right vhost in name based vhosting
On 2013-09-24 11:38, Christoph Gröver wrote: Hello list, Hello Sorin, I tested several different Apaches (2.4.x and 2.2.x) and they never did the wanted or expected. If I configure more than one VHost only the first one is returned by the server-server_hostname structure. The one of the second vhost that is configured as a ServerName seems to be impossible to determine? Is there any other way to find the hostname? I suppose you use the server field of the request_rec structure and not some stored server_rec that was passed to you in post_config or somewhere else. Apache keeps a linked list of server_rec structures. The head of the list is the server_rec of the whole apache server. The rest of the list contains one server_rec structure per vhost. For each request apache picks the right server_rec from the list according to the Host header and sets r-server to point to the picked object. Also make sure that your request really arrives in the vhost you intended. Typically I check this by logging to different files (see the CustomLog directive) in each vhost. Regards, Sorin
Re: How to determine the right vhost in name based vhosting
On 2013-09-24 13:04, Christoph Gröver wrote: Hello Sorin, I suppose you use the server field of the request_rec structure and not some stored server_rec that was passed to you in post_config or somewhere else. Definitely. I have adopted this from some other module and didn't know there was another way to obtain a server_rec structure. So I should be looking for a better way to find the right structure. Thank you very much. This sounds as if it will be the right way. I fear there's a misunderstanding here: The right way to get the server_rec is, in my opinion, from the request_rec structure, i.e. I think you should use req-server-server_hostname. So, given that you already do this, it is puzzling for me why you don't get the result that you want. Apache sets the req-server pointer to the right server_rec structure after it has parsed the request headers. (It cannot guess correctly before it parses the Host header.) So make sure you check req-server _after_ apache has initialised it to the right server_rec. Apache sets it in the ap_read_request method. Almost all of the callbacks provided to the module developers are called _after_ ap_read_request, so you should be ok. I think only the create_connection callback is run before ap_read_request. As a poor man's debugger technique you could write a post_config callback. The last argument of the post_config callback is the head of the list of server_recs. You could traverse the list and log to a file the server_hostname of all server_recs in the list. Just to check that you have the right number of server_recs and that they are correctly initialised. Sorin Apache keeps a linked list of server_rec structures. The head of the list is the server_rec of the whole apache server. The rest of the list contains one server_rec structure per vhost. For each request apache picks the right server_rec from the list according to the Host header and sets r-server to point to the picked object. This information will also help. Thank you. Also make sure that your request really arrives in the vhost you intended. Typically I check this by logging to different files (see the CustomLog directive) in each vhost. This is actually the case. I receive the requests in the right vhost. I have separate logfiles for each vhost. Thanks for your answers. I guess I will be able to solve the issue with these informations. With kind regards,
Re: Browser cookie and Apache server calls
On 2013-06-27 06:28, Sindhi Sindhi wrote: If I clear the browser cache before I click on the hyperlink, I dont see this issue. But I do not want to delete the cookies, because the business logic used by the filter makes use of the cookies that are set. Also I may not want to delete the cache everytime before I click on the hyperlink :( I added the below lines in httpd.conf file but still see that the page is cached and hence no server call is made :( LoadModule headers_module modules/mod_headers.so Header set Cache-Control must-revalidate, no-cache As a general advice, test your modules with a command line tool first. Thus you have a strict control of what you send in your request and you see what the server answers. Such a command line tool is curl. It runs under Windows too. It allows you to locate the problem: is it that your module does not send the expected headers (Set-Cookie, Cache-Control, etc), or is it that the browser does not send them (the Cookie header for example). With curl you can specify which headers to send, which cookies, and you can simulate browsers by sending all kind of cache-related headers (If-Modified-Since, If-Match-None, etc). Check http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14 for an explanation of all http headers, especially the cache-related ones (Cache-Control, Expires, Last-Modified, If-Modified-Since, ETag, etc). For testing with the browser, there's a plugin for Firefox called Firebug. Maybe there's something similar for Chrome, I don't know. Firebug displays the http communication between browser and server, including the http headers. If you don't find such a plugin, you could sniff the network traffic directly via a tool such as wireshark (works on Windows too and it's free). You can look if the Cache-Control header is always set by your server. If it is set then the browser is not supposed to cache the page linked from the hyperlink, so it should replay the request every time you click on it. If it is not set and you cannot force apache to set it by the Header directive, then force it directly in your module (apr_table_set(r-headers_out, Cache-Control, must-revalidate, no-cache); apr_table_set(r-err_headers_out, Cache-Control, must-revalidate, no-cache);) Sorin On Thu, Jun 27, 2013 at 2:25 AM, Sorin Manolache sor...@gmail.com wrote: On 2013-06-26 22:22, Sindhi Sindhi wrote: Hi, I have a C++ Apache filter module for HTML output filtering. I'm seeing a certain behavior when using my filter module after setting cookies and would want to know if thats expected. I have a html page index.html, this page has a hyperlink called Click here and when I click on this link, a second page index2.html should be launched. My filter applies some business logic while filtering html files. I have a cookie.html file that has Javascript to set a cookie (using document.cookie) in the browser. I need to do the following: 1. Enable my filter using LoadModule directive and start the server 2. Set a cookie with name=cookieName, value=cookieValue in the browser using the cookie.html 3. Launch index.html, and then click on Click here. When this call goes to the filter module, I have to apply some business logic before index2.html is rendered on browser. But when I set the cookie in step2 above, I see that the filter module is not called because a server call is not made, and the browser opens the cached index2.html which does not have my business logic applied. And, if I dont set the cookie mentioned in step2 above, a server call is made when I click on Click here link. How can I ensure that, when I try to launch a html page from a hyperlink, the call goes to the filter module even when I set a browser cookie. What happens if you clear your browser's memory and disk cache before you click on the hyperlink? If it's a cache issue, then use the mod_headers module and the 'Header set Cache-Control must-revalidate, no-cache' directive to disable browser caching. Sorin My apologies if I'm asking something fundamental, I'm new to how cookies work with web-servers, any help would be really appreciated. Thanks.
Re: Browser cookie and Apache server calls
On 2013-06-26 22:22, Sindhi Sindhi wrote: Hi, I have a C++ Apache filter module for HTML output filtering. I'm seeing a certain behavior when using my filter module after setting cookies and would want to know if thats expected. I have a html page index.html, this page has a hyperlink called Click here and when I click on this link, a second page index2.html should be launched. My filter applies some business logic while filtering html files. I have a cookie.html file that has Javascript to set a cookie (using document.cookie) in the browser. I need to do the following: 1. Enable my filter using LoadModule directive and start the server 2. Set a cookie with name=cookieName, value=cookieValue in the browser using the cookie.html 3. Launch index.html, and then click on Click here. When this call goes to the filter module, I have to apply some business logic before index2.html is rendered on browser. But when I set the cookie in step2 above, I see that the filter module is not called because a server call is not made, and the browser opens the cached index2.html which does not have my business logic applied. And, if I dont set the cookie mentioned in step2 above, a server call is made when I click on Click here link. How can I ensure that, when I try to launch a html page from a hyperlink, the call goes to the filter module even when I set a browser cookie. What happens if you clear your browser's memory and disk cache before you click on the hyperlink? If it's a cache issue, then use the mod_headers module and the 'Header set Cache-Control must-revalidate, no-cache' directive to disable browser caching. Sorin My apologies if I'm asking something fundamental, I'm new to how cookies work with web-servers, any help would be really appreciated. Thanks.
Re: Mutex protection of output bucket brigade
On 2013-06-12 11:48, Alex Bligh wrote:
On 12 Jun 2013, at 10:20, Sorin Manolache wrote:
If I understand correctly, the main thread belongs to your module, i.e. it is
not a concise pseudo-code of the request processing in apache's code.
The main thread is the (presumably single) thread of the prefork mpm process,
created (I assume) by a fork() in apache's worker code.
The pseudo code was what my long-running request handler does (after creating
the other thread). IE, broadly speaking my request handler (the main thread if
you like) does this:
apr_thread_create; /* create the spawned thread */
while (!done)
{
/* Blocking read */
apr_brigade_create;
ap_get_brigade;
apr_brigade_flatten;
/* Do stuff with the data */
blocking_socket_write;
}
apr_thread_join; /* wait until the spawned thread has executed */
I don't see where the output brigade appears in the main thread. I think this
is critical, as the output_bucket_brigade is the data item shared between the
two threads. ap_get_brigade triggers the execution of the chain of input
filters. One of these input filters writes to the output brigade?
ap_get_brigade is called with APR_BLOCK_READ.
What I now /believe/ this does (because it's the only way data would actually
get written) is write the post processed output bucket brigade to the client
too. If this were not the case, it's difficult to see how a single threaded
application would every write the output bucket brigade.
Or are you saying the output bucket brigade is only actually written to the
client during an ap_fwrite()? In which case are all the filters (primarily
mod_ssl) guaranteed to be thread safe if a different thread is doing input from
that doing output?
Normally the output brigade is only written during the
ap_rprintf/ap_fwrite and the like.
There is no output brigade when the request_rec structure is created.
The way to write something to the client is via
ap_pass_brigade(r-output_filter, brigade). Typically the function that
calls ap_pass_brigade creates the brigade first. So you write something
to the brigade and you pass it downstream. The last filter writes its
contents to the network. Be aware that there are filters that buffer the
brigades and do not send them further down the chain unless the buffers
are full.
If you created a handler that would just return OK without ever calling
an ap_rprintf/ap_fwrite, the module would create no output brigade and
the client would get no response.
I think your hypothesis is unlikely. For it to hold, you'd need an input
filter that is triggered when ap_get_brigade is called, that that input
filter creates a brigade and stores it somewhere and that the spawned
thread somehow writes to this stored brigade. So the ap_fwrite in the
spawned thread would need to write to a brigade created in an input
filter triggered by ap_get_brigade in the main thread.
To verify this hypothesis, check which is the brigade whose pointers are
corrupted. I mean, where is it referenced, in which apache module, in
which apache filter. Then you can inspect the sources of that module to
see if the brigade is shared between input and output.
S
Re: Mutex protection of output bucket brigade
On 2013-06-12 14:16, Alex Bligh wrote: But that aside, is it safe to call apwrite() from one thread whilst there's a read in the other? I do not know for sure, but I suppose it is not safe if both the read and the write operate on the same brigade. I'm not calling ap_pass_brigade (at least not directly). I'm doing (roughly) ap_filter_t *of = state-r-connection-output_filters; ap_fwrite(of, state-obb, (const char *)header, pos); /* Header */ IE I'm doing an fwrite to output filter list, using the bucket brigade I've just created. Is that OK? I've checked the code of ap_fwrite. Apparently it buffers the data in the brigade. If the brigade is full, it calls ap_filter_flush, which, in turn, calls the ap_pass_brigade and clears the brigade after ap_pass_brigade has returned. So not every call to ap_fwrite will push the data down the filter chain. Now I think I understand what happens with ap_fwrite (if that calls ap_pass_brigade), I'm wondering whether (say) the input filter of mod_ssl ever talks to its output filter. I think it's impossible. The brigade is created by you in the spawned thread, so I don't see how the SSL decode in the main thread could access it. And it's the output brigade that is corrupted, not some brigade internal to mod_ssl. As I see it, the output brigade (state-obb) is not shared between the threads. Do you get the same errors when you disable mod_ssl? S
Re: thread safety and mpm-prefork
On 2013-06-11 21:20, Alex Bligh wrote: I've written a module which I believe to be thread-safe but appears to be doing something which I have put down to a lack of thread safety in pool management (somewhere). Before I tear my hair out here, my module is running with apache 2.2.22 and mpm-prefork on Ubuntu. Do the thread primatives actually do anything in mpm-prefork? I'm using apr_thread_create to create a thread, then providing a separate allocator, mutex, pool and similar (all as recommended). But if the mutex stuff is 'optimised out' of my apr library - specifically the pool stuff - all this will be in vain. apr_* and mpm_prefork are different software packages and ubuntu distributes them separately. So it is almost certain that you have a thread-enabled libapr (i.e. compiled with APR_HAS_THREADS). You would not be able to compile the code that uses apr_thread_create if your libapr was not compiled with thread support. mpm_prefork is like any ordinary client of libapr. Just that it does not use the threading functionality in libapr. So it cannot disable/optimise out the mutexes in libapr. Please be aware that apr_pools are not thread-safe. Only the creation of subpools is thread-safe. So you should create a subpool per thread to stay safe. Sorin
Re: Apache pool management
On 2013-05-25 10:05, Sindhi Sindhi wrote: You have answered all my questions and thanks a lot. Had two questions more, appreciate your response. 1. As of now, my httpd.conf file has the below lines- # Server-pool management (MPM specific) #Include conf/extra/httpd-mpm.conf This means Apache does not read the httpd-mpm.conf file during startup. And so it uses the default settings for Max number of requests it can support and Max number of threads it creates per child process Where can I find what default values Apache uses for the following - - Upto how many concurrent requests will Apache support by default - Max number of threads that one child process creates For ex. if I want Apache to handle upto 400 concurrent requests at a time, how will I know that this 400 is within the default settings that Apache uses. Have a look here, depending on your version of apache. http://httpd.apache.org/docs/2.2/mod/mpm_common.html http://httpd.apache.org/docs/2.4/mod/mpm_common.html The number of threads per process is given in ThreadsPerChild. You typically tweak ServerLimit (the maximum number of children that are simultaneously alive) ThreadLimit (the maximum sum of alive threads in all children) StartServers (how many children are created upon startup) ThreadsPerChild MaxRequestWorkers (or MaxClients in 2.2) (the maximum number of requests that are served simultaneously) MaxConnectionsPerChild (or MaxRequestsPerChild in 2.2) (after a child has served that many requests, it exits and is potentially replaced with a new child; avoids memleaks) MinSpareThreads and MaxSpareThreads, the minimum and maximum number of spare threads. There are some constraints on the arguments of these directives, which I do not master. I think that MaxRequestWorkers = ThreadsPerChild * ServerLimit and that MaxRequestWorkers and ThreadLimit should be divisible by ThreadsPerChild, but as I said, I do not master. If you get them wrong, apache adjusts the values automatically and informs you about it upon startup. I am not sure, maybe others on the list can confirm or deny, but I think that apache does not distinguish between threads and processes in windows: http://httpd.apache.org/docs/2.4/mod/mpm_winnt.html or http://httpd.apache.org/docs/2.2/mod/mpm_winnt.html. So I think that ServerLimit = 1 in Windows and probably MaxConnectionsPerChild is not used or does not exist. You may also have a look at KeepAlive On|Off, MaxKeepAliveRequests and KeepAliveTimeout (http://httpd.apache.org/docs/2.4/mod/core.html) If my module is on the internet with hundreds of thousands of possible client IPs that issue one request and then leave, I set KeepAlive Off. If my module is on the intranet and is accessed by a couple of webservices that continuously issue requests, I set it On with a short timeout. Performance-wise the KeepAlive directive makes a huge difference. 2. My understanding is, once a request is completely processed, Apache frees the pool of only this request and does not free any other request's pool. And other request pools will be freed only when those requests are completely processed. Kindly confirm my understanding to be correct. Yes, it is correct. Sorin
Re: Get name of config file for module
On 2013-05-21 23:52, Sean Beck wrote: Sorin, Is there a way to figure out the name of the config file in code so I can log it? Or even just the path to where it is. Also, I'm confused because you said there is no such thing as a module-specific configuration file, but then you said configuration files can be split per-module. Does Apache just read out of httpd.conf on start-up and in httpd.conf I would use the Include directive to include my different configurations for each module? Yes, exactly. Apache knows its config file, either because it's hard-coded when apache was compiled, or because the hard-coded path was overwritten by command line arguments (the -f switch). Then, the root config file, so to say, contains Include directives to other files. But it's only syntactic sugar for conceptually separating configurations for the benefit of the server administrator. Apache has no way of mapping the included files to modules. I don't think there is a way to get the name of the included files. And I don't think there is a way either to get the name of the root configuration file, as far as I---briefly---looked in apache's sources. Sorin Thanks On Tue, May 21, 2013 at 2:32 PM, Sorin Manolache sor...@gmail.com wrote: On 2013-05-21 21:36, Sean Beck wrote: Hi all, I have written a module and now would like to log the name of the actual config file being used by the module (there will be multiple modules on the server with their own config files. I looked through https://httpd.apache.org/docs/**2.4/developer/modguide.htmlhttps://httpd.apache.org/docs/2.4/developer/modguide.htmlbut am still struggling to understand how Apache knows what the config file is for each module and how I can check what the name of the actual name of the file is for each module. There is no such thing as a module-specific configuration file. In principle a single file could contain the configuration for all modules. The configuration files are split per-module for convenience only, in order to be able to activate modules independently. The configuration _directives_ are defined per-module (but actually nothing stops you from defining the same configuration directive in several modules). Apache defines a directive called Include. This directive allows for the splitting of the entire configuration into several files. But from apache's point of view there is no correspondence between the files into which the configuration is split and the modules that define the directives found in those files. Sorin
Re: C++ Apache module fails to load
On 2013-05-11 08:35, Sindhi Sindhi wrote: Hello, I have created a C++ Apache module that performs text filtering operations. I used Visual Studio 2010 to build/compile the source code to generate the DLL. I have Apache httpd-2.4.4-win64 installed from http://www.apachelounge.com/download/win64/. I have a Windows 7 64-bit OS. Loading and executing this module works absolutely fine on my laptop. But if I copy the same DLL on to a different laptop having same operating system and Apache configuration, I get the following error when I try to start httpd.exe - httpd.exe: Syntax error on line 174 of C:/Apache24/conf/httpd.conf: Cannot load modules/MyFilter.dll into server: The specified module could not be found. Could you please advice? I have no clue but check that you have exactly the same configuration and the same directory tree. Check the arguments of the LoadModule directive and check that MyFilter.dll is placed where LoadModule expects it. Try replacing the relative paths with absolute paths. The relative paths are relative to the ServerRoot, so check that the ServerRoot directive has the same arguments. Check that the ServerRoot or other configuration directives are not overwritten by some command-line arguments in the apache-lauching script, if any. Sorin
Re: Apache C++ equivalent of javax.servlet.Filter
On 2013-05-01 12:21, Sindhi Sindhi wrote:
Hi,
I'm developing a C++ module for Apache(httpd.exe) server. This C++ module
intends to function the same as a Java module that we earlier developed for
Tomcat. I'll be very thankful to you if you could answer the following
queries I have about finding the Apache(httpd.exe server) C++ equivalents
for the below Java classes.
This Java code has references to the following -
1. javax.servlet.Filter
In Java we have a class CustomFilter that implements javax.servlet.Filter.
This CustomFilter class has an init() method that will be called when
Tomcat starts. How can I achieve this for Apache(httpd.exe server)? means,
how can I make a function call when Apache(httpd.exe server) starts.
There are three possibilities:
1. the pre_config hook. This is invoked before apache parses its
configuration. I think this is not what you want.
2. the post_config hook. This is invoked after apache parses its
configuration but before the children processes are invoked. This could
be what you want.
pre_config and post_config are called twice when apache starts and each
time the configuration is reloaded.
pre_config and post_config are called in the apache parent process with
the privileges of the apache parent process.
3. the child_init hook. This is invoked whenever apache creates a new
child process. This is invoked with the privileges of the apache
children processes.
That was the answer to your question. However, in apache filters are
typically initialised differently. Each filter may have an init
function. If the filter has such an init function (i.e. if it's not
null), then the init filter function is invoked automatically by apache
after the fixups callback and before the handler callback. So the init
function, if it exists, is invoked once per request.
2. javax.servlet.FilterConfig
The above mentioned init() method takes in an argument of type
FilterConfig. What is the Apache C++ equivalent of FilterConfig?
The ap_filter_t structure has a void pointer field called ctx. This is
null by default. You can make it point to whatever object you want. You
can initialise this ctx in the init function of the filter or when the
filter is invoked for the first time. (Please note that a filter may be
invoked several times for the same request, so you'll have to take care
to distinguish between the several invocations for the same request and
between the invocations triggered by different requests.)
I'm not sure what's the role of the FilterConfig object in Java
servlets. Maybe you need a configuration object of your apache module
and not a filter context. A filter context serves mainly to store a
state between consecutive invocations of the filter for the same
request. So the data in the filter context, as it is a state, changes.
It's not really a configuration, which is more a read-only object.
3. The interface javax.servlet.Filter also has the method
doFilter(ServletRequest request, ServletResponse response, FilterChain
chain). In Apache C++ I can get the filter chain using the structure
ap_filter_t. But how will I get the objects of
ServletRequest/HttpServletRequest and ServletResponse/HttpServletResponse
in C++ module? Means what are the corresponding structures I can access in
Apache C++ module.
The main filter callback function in my C++ module looks like this -
EXTERN_C_FUNC apr_status_t filterOutFilter (
ap_filter_t *filterChain,
apr_bucket_brigade *inBucketList)
The request_rec *r field of the ap_filter_t structure plays the role of
the ServletRequest.
The ap_filter_t *next field of the ap_filter_t structure plays the role
of the FilterChain.
There is no ServletResponse object. Your filter gets this bucket
brigade which is basically a linked list of buckets. The buckets
contain data coming from the upstream filters or from the handler.
So a typical filter would parse the bucket brigade (i.e. traverse the
linked list of buckets and process the data they contain) and generate a
new, filtered (transformed) bucket brigade. Then it would pass it to
downstream filters.
Something like
for (all buckets in input brigade) {
read data in bucket
transform the data, optionally using
the state stored in f-ctx
optionally update the state in f-ctx
append the transformed data to the output brigade
if (we reached the end-of-stream)
// if upstream filters were well-behaved
// this would be the last invocation of the filter
// for this request
return ap_pass_brigade(f-next, output_brigade)
}
// we parsed the whole brigade without reaching
// to the end-of-stream = the filter will be invoked again
// later with the next part of the data coming from upstream
// filters
return ap_pass_brigade(f-next, output_brigade);
Note, as I said previously, that the filter may be called several times
for the same request.
After passing a brigade containing an EOS (end-of-stream)
Re: Apache Buckets and Brigade
On 2013-05-01 14:54, Sindhi Sindhi wrote:
Hello,
Thanks a lot for providing answers to my earlier emails with subject
Apache C++ equivalent of javax.servlet.Filter. I really appreciate your
help.
I had another question. My requirement is something like this -
I have a huge html file that I have copied into the Apache htdocs folder.
In my C++ Apache module, I want to get this html file contents and
remove/replace some strings.
Say I have a HTML file that has the string oldString appearing 3 times in
the file. My requirement is to replace oldString with the new string
newString. I have already written a C++ function that has a signature
like this -
char* processHTML(char* inHTMLString) {
//
char* newHTMLWithNewString = code to replace all occurrences of
oldString with newString
return newHTMLWithNewString;
}
The above function does a lot more than just string replace, it has lot of
business logic implemented and finally returns the new HTML string.
I want to call processHTML() inside my C++ Apache module. As I know Apache
maintains an internal data structure called Buckets and Brigades which
actually contain the HTML file data. My question is, is the entire HTML
file content (in my case the html file is huge) residing in a single
bucket? Means, when I fetch one bucket at a time from a brigade, can I be
sure that the entire HTML file data from html to /html can be found in
a single bucket? For ex. if my html file looks like this -
html
..
..
oldString
... oldString...oldString..
..
/html
When I iterate through all buckets of a brigade, will I find my entire HTML
file content in a single bucket OR the HTML file content can be present in
multiple buckets, say like this -
case1:
bucket-1 contents =
html
..
..
oldString
... oldString...oldString..
..
/html
case2:
bucket-1 contents =
html
..
..
oldStr
bucket-2 contents =
ing
... oldString...oldString..
..
/html
If its case2, then the the function processHTML() I have written will not
work because it searches for the entire string oldString and in case2
oldString is found only partially.
Unfortunately there is no guarantee that the whole file is in one brigade.
Even if it was in one brigade, there is no guarantee that is it in a
single bucket.
So you can have case2.
In my experience the buckets that I've seen have about 8 kilobytes. So
you will not consume too much memory if you flatten the bucket brigade
into one buffer and then perform the replacement in the buffer. (see
apr_brigade_flatten). However, you have to provide for the case in which
oldString is split between two brigades.
Sorin
Thanks a lot.
Re: retrieving module configs from other modules
On 2013-04-15 23:12, Tim Traver wrote: Hi all, is there a way to access the module config tables of another module? For instance, if I want to see (from my module) what the path to the SSL cert is in the ssl_module, how can I go about doing that? And are there any tutorials on how to retrieve and set config values? Thanks, Tim I think you have two options: 1) You handle exactly the same configuration directive in your module in order to intercept what other modules see. This is possible because apache invokes the configuration handling callbacks of all modules that have registered for that directive. So in theory your module can intercept the whole configuration of apache and all its modules. 2) You read the configuration object of the other module. However you'll need two pieces of information in order to do so: The name of the other's module object and the structure of its configuration object. For example you'd do: OtherModConf *conf = (OtherModConf *)ap_get_module_config(conf_vector, other_mod_obj); use(conf-conf_value_of_interest); I think you cannot know other_mod_obj unless you search in the other module's sources. (You could check the exported symbols of the binary as well.) And you may know OtherModConf only if the other module publishes its structure in a header file intended to be included in modules such as yours. Sorin
Re: How to control block size of input_filter data
On 2013-03-12 10:52, Hoang-Vu Dang wrote:
Thank you for the quick reply !
The context is what I am looking into right now, and It is indeed a
right solution to my original question. I just want to know a little bit
more detail if you do not mind, you said:
I typically destroy it by placing a callback in the cleanup hook of the
req-pool.
Now I remember: I use C++ so I need to create and destroy the context.
But if you allocate your context from an apr_pool you don't need to
bother about the context destruction because it is automatically
destroyed. Sorry for confusing you.
Just for information, I create/destroy the contexts like that:
int
flt_init_function(ap_filter_t *flt) {
// I use C++, if you allocated ctx from a pool, you don't even need
this callback destruction
flt-ctx = new MyContext();
// delete_ctx is called when r-pool is destroyed, i.e. at the very
end of the request processing, after the response has been sent to the
client and the request logged.
apr_pool_cleanup_register(flt-r-pool, flt-ctx, (apr_status_t
(*)(void *))destroy_ctx, apr_pool_cleanup_null);
}
and
apr_status_t
destroy_ctx(MyContext *ctx) {
delete ctx;
return APR_SUCCESS;
}
The filter function could be something like:
apr_status_t
input_filter(ap_filter_t *f, apr_bucket_brigade *bb, ap_input_mode_t
mode, apr_read_type_e block, apr_off_t bytes) {
MyContext *ctx = (MyContext *)f-ctx;
switch (ctx-state()) {
case FIRST_INVOKATION:
...
break;
case NTH_INVOKATION:
...
break;
case FOUND_EOS:
...
break;
...
}
}
What exactly is the callback function that I need to look for ? When it
executes, can we be sure that all the data has been processed, and our
ctx will be maintained at that state ?
Best, Vu
On 03/12/2013 10:36 AM, Sorin Manolache wrote:
On 2013-03-12 10:16, Hoang-Vu Dang wrote:
Hi all,
When I write an input_filter, I notice that the data sent from client is
not always available in one chunk if it's large.
In other words, The input_filter() function will be called multiple
times per request. My question is how to have a control on this (for
example the size of the chunk until it breaks in to two) ? what should
we look into in order to check if the two filters are called from the
same request.
You can keep the state from one filter invokation to the other in
f-ctx, the filter's context.
There are many ways to do this.
One way I've seen is to check if f-ctx is NULL (if it was NULL then
it would mean that it is the first invokation of the filter). If it's
NULL, we build the context. Subsequent invokations have the context !=
NULL. You'll have to destroy the context at the end of the request. I
typically destroy it by placing a callback in the cleanup hook of the
req-pool.
Another way to destroy it, but in my opinion a wrong way, is to
destroy it when you encounter EOS in the data processed by the filter.
I'd say it's wrong because a wrongly written filter could send data
_after_ an EOS bucket and then you could not distinguish between a new
request and a request sending data after EOS.
Another way to initialize the context is by placing a filter init
function when you declare the filter and to initialize the context in
this function. This is more elegant in my opinion, because the context
is already initialized when the filter is called the first time.
The filter context could be any structure, so you can track the filter
processing state in it.
Regards,
Sorin
Re: Read file file before web server startup
On 2012-11-13 00:41, Idel Fuschini wrote: Hi, I need to read a configuration file (XML) and put dati in hash array before the webserver starts. It's possibile, if yes how ? You define a module configuration object using the create_server_config callback of your module structure. You define a configuration directive that specifies the name of the XML file (see the command_rec array of the module structure). In the command_rec array entry that defines your configuration directive, you associate the configuration directive with a callback function. The callback function is called when apache encounters the directive. The argument of the configuration directive (in your case the XML file name) is passed by apache as an argument to the callback function. You get the configuration object in the configuration directive callback using the ap_get_module_config function. You read the XML file in the callback function and initialise the configuration object. Sorin
Re: Forcing Apache to exit on startup
On 2012-10-22 21:29, Joshua Marantz wrote: Hi, Our module has multiple confirmation parameters. There is a case where if you have option A and option B, then you must also specify option C, otherwise Bad things can happen that are a lot easier to debug on startup than they are after the server is running. I know how to force 'apachectl' to exit with a nice error message if I don't like the value of option A or option B. I can do that in the function-pointer I provide in my option-table by returning a non-null char* message. But in this case I want to exit nicely from the function I have registered with ap_hook_child_init, having noticed that option A and B are set but not C. Is that possible? By nicely, I mean that the user types: % sudo /etc/init.d/apachectl restart Error in pagespeed.conf: if you have option A and B specified, you must specify a value for option C. At that point either the server would not be running, or it would still be in whatever state it was previously in. Is this possible? Currently our solution is to log an error and call abort(), and it's not very nice! Don't do it in child_init. I don't think the child can stop the server. I think apachectl spawns apache with an appropriate -k argument (e.g. apache2 -k stop) to start/stop/reload apache. However, I don't think that the child has sufficient privileges to stop the parent by exec'ing apache2 -k stop. Nor can it send the apropriate signal to the parent. But you can exit nicely in post_config, which is run (as root) after the conf is read but before the children are spawned. It suffices to return a non-ok code for apache to exit. I _think_, I'm not sure, that what you log in post_config does not go on the console but goes to the server log. But maybe there's a way to put the error message on the console too. Sorin
Re: Close HTTP connection callback/hook
On 2012-10-16 15:50, Evgeny Shvidky wrote: Hi, I am implementing a new module on C. I need to perform some functionality when a user closes a HTTP connection before he received any response for his request. How can I know when a HTTP user request state has been changed/closed? Is there any callback/hook for this functionality I can register? As far as I know, there is no callback for this functionality. You can check r-connection-aborted or the APR_ECONNABORTED return code of ap_pass_brigade when you attempt to write to the client. Sorin
Re: Modules Communicating
On 2012-08-22 09:31, Adi Selitzky wrote:
Hi!
I am writing my own module that handles all incoming requests. In some cases, I
want this request to be handled by mod_python which I installed. In these
cases, my module should change the requested file extension to .py, which is
configured to be handled by mod_python.
AddHandler mod_python .py
I have tow questions:
1. How can I set the modules order, so my module will handle the request first,
change its url, and then mod_python will handle it?
const char *succ[] = {mod_python.c, NULL};
ap_hook_handler(your_handler, NULL, succ, APR_HOOK_FIRST);
2. Which field in the request_rec I should change so it will take effect? I
tried to change the URL key in subprocess_env table, but the request was not
handled by mod_python.
AddHandler mod_python .py simply sets r-handler to mod_python if
r-filename contains .py. So I guess that the python processing is
triggered by r-handler being mod_python.
So you can try setting r-handler = mod_python and then return
DECLINED from your handler and forget about appending .py.
Sorin
Re: How to access string in a module that is set in configuration directive?
On 2012-06-30 22:33, oh...@cox.net wrote:
Hi,
I got my 1st module working and now I need to add support for a configuration directive
that sets a string that my module needs in various places, i.e., if my directive is
SetMyVar, and httpd.conf has:
SetMyVarfoo123
then, in my module, I need to able to access the string, foo123, that got set via the
SetMyVar directive. This directive would only be used at the server level.
I think that I know how to do the code to set the variable into my module, but
the question that I have is how do I *access* it after it is set?
Here's the code I have to handle the directive:
/*
* Stuff for Directive handling
*/
// Struct for Directives
typedef struct txt_cfg {
const char * MyVar;
} txt_cfg;
// Function to get GxInterceptorURL
static const char * txt_set_MyVar(cmd_parms* cmd, void* cfg, const char* val) {
printf(In set_MyVar: Setting MyVar to [%s]\n, val);
((txt_cfg*) cfg)-MyVar = val;
} // end txt_set_MyVar()
//
static const command_rec txt_cmds[] = {
AP_INIT_TAKE1(SetMyVar, txt_set_MyVar, NULL, OR_ALL,
This is MyVar),
{ NULL }
};
.
.
.
module AP_MODULE_DECLARE_DATA my_module =
{
STANDARD20_MODULE_STUFF,
NULL, /* dir config creater */
NULL, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
txt_cmds, /* command apr_table_t */
register_hooks /* register hooks */
};
Can anyone tell me how, in my module code, I can access that MyVar string?
You don't get a valid configuration object (txt_cfg) in the cfg argument
of the txt_set_MyVar unless you create that configuration object. As
your code looks now, cfg in txt_set_MyVar should be null.
You'll have to set a callback function for creating the configuration
object. The configuration object configuration functions are
dir_config_creater (first after STANDARD20_MODULE_STUFF in my_module)
and server_config (third after STANDARD20_MODULE_STUFF).
The dir_config creator will put your configuration object in
r-per_dir_config and you get it with
txt_cfg *cfg = (txt_cfg *)ap_get_module_config(r-per_dir_config,
my_module);
The server_config_creator will put your configuration object in
r-server-module_config and you get it with
txt_cfg *cfg = (txt_cfg *)ap_get_module_config(r-server-module_config,
my_module);
Have a look in include/http_config.h at all those OR_* macros and
especially at RSRC_CONF and ACCESS_CONF.
RSRC_CONF is used mainly to define server-wide directives. ACCESS_CONF
is used to define directory-wide configuration directives.
In your cfg argument of the txt_set_MyVar configuration directive
handler, you'll get the _directory-wide_ configuration object (if you've
created one by using the dir_config_creator in module my_module). If you
want to set your variable just for the directory, you can keep
txt_set_MyVar as it is now. Later in the module, you can retrieve the
value as I showed above, i.e. from r-per_dir_config.
If you want to set it server-wide, then don't use cfg. Write
txt_cfg *srv_cfg = (txt_cfg
*)ap_get_module_config(cmd-server-module_config, my_module).
Later in the module, you can retrieve the value as I showed above, i.e.
from r-server-module_config.
Remember that in any case, you have to create the respective
configuration object (via the callbacks in my_module).
S
Thanks,
Jim
Re: Confused about modules processing order...
On 2012-06-26 03:49, oh...@cox.net wrote:
Hi,
I have my small prototype module, which I implemented starting with the
mod_headers.c source, working. The changes that I did to the original source
were to add some code in the insert_filter hook to inject an additional header
into the request.
That seems to work ok with a vanilla Apache configuration.
I want to be able to make my modified module work together with another module, provided
by Oracle (the Oracle Access Manager webgate, aka webgate).
However, after I add the directives into the Apache httpd.conf to enable the
webgate, it appears that, on incoming requests, the webgate processing occurs,
but my code in the modified mod_headers module is not called at all :(!!
Here's the last part of my modified mod_headers.c:
static void register_hooks(apr_pool_t *p)
{
printf(mod_headers-jl V0.13 - use LIBCURL instead of OAM ASDK-process response
from callCurl\n);
printf(In register_hooks\n);
ap_register_output_filter(FIXUP_HEADERS_OUT, ap_headers_output_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_register_output_filter(FIXUP_HEADERS_ERR, ap_headers_error_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_insert_filter(ap_headers_insert_output_filter, NULL, NULL,
APR_HOOK_LAST);
ap_hook_insert_error_filter(ap_headers_insert_error_filter,
NULL, NULL, APR_HOOK_LAST);
ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
ap_hook_post_read_request(ap_headers_early, NULL, NULL, APR_HOOK_FIRST);
}
module AP_MODULE_DECLARE_DATA headers_module =
{
STANDARD20_MODULE_STUFF,
create_headers_dir_config, /* dir config creater */
merge_headers_config, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
headers_cmds, /* command apr_table_t */
register_hooks /* register hooks */
};
The code I added is in the ap_headers_insert_output_filter() function.
I did an export SHOW_HOOKS=1 and ran the Apache, and I see this for the
modified mod_headers:
Registering hooks for mod_headers.c
mod_headers-jl V0.13 - use LIBCURL instead of OAM ASDK-process response from
callCurl
In register_hooks
Hooked pre_config
Hooked post_config
Hooked insert_filter
Hooked insert_error_filter
Hooked fixups
Hooked post_read_request
And for webgate, I see:
Registering hooks for apache2entry_web_gate.cpp
Hooked post_config
Hooked handler
Hooked check_user_id
Hooked auth_checker
I thought that the handler functions are called almost last part of the
processing (content generation), and my code is hooked to insert_filter, which
I thought occurs earlier than content generation, so shouldn't my code get
processed BEFORE Apache attempts to process the webgate functions?
How can I get my code to process before the webgate does?
insert_filter is run between the fixups and the handler hooks.
Try to identify who is producing the variables that you need, in which
phase they are available at the earliest. Then identify which part of
web_gate hijacks the processing such that your code is not executed
anymore. I suppose it is one of web_gate's auth_checker or
check_user_id. If it was the web_gate handler then your code would have
run before.
Sorin
Re: Confused about modules processing order...
On 2012-06-26 13:14, oh...@cox.net wrote:
Sorin Manolache sor...@gmail.com wrote:
On 2012-06-26 03:49, oh...@cox.net wrote:
Hi,
I have my small prototype module, which I implemented starting with the
mod_headers.c source, working. The changes that I did to the original source
were to add some code in the insert_filter hook to inject an additional header
into the request.
That seems to work ok with a vanilla Apache configuration.
I want to be able to make my modified module work together with another module, provided
by Oracle (the Oracle Access Manager webgate, aka webgate).
However, after I add the directives into the Apache httpd.conf to enable the
webgate, it appears that, on incoming requests, the webgate processing occurs,
but my code in the modified mod_headers module is not called at all :(!!
Here's the last part of my modified mod_headers.c:
static void register_hooks(apr_pool_t *p)
{
printf(mod_headers-jl V0.13 - use LIBCURL instead of OAM ASDK-process
response from callCurl\n);
printf(In register_hooks\n);
ap_register_output_filter(FIXUP_HEADERS_OUT, ap_headers_output_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_register_output_filter(FIXUP_HEADERS_ERR, ap_headers_error_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_insert_filter(ap_headers_insert_output_filter, NULL, NULL,
APR_HOOK_LAST);
ap_hook_insert_error_filter(ap_headers_insert_error_filter,
NULL, NULL, APR_HOOK_LAST);
ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
ap_hook_post_read_request(ap_headers_early, NULL, NULL, APR_HOOK_FIRST);
}
module AP_MODULE_DECLARE_DATA headers_module =
{
STANDARD20_MODULE_STUFF,
create_headers_dir_config, /* dir config creater */
merge_headers_config, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
headers_cmds, /* command apr_table_t */
register_hooks /* register hooks */
};
The code I added is in the ap_headers_insert_output_filter() function.
I did an export SHOW_HOOKS=1 and ran the Apache, and I see this for the
modified mod_headers:
Registering hooks for mod_headers.c
mod_headers-jl V0.13 - use LIBCURL instead of OAM ASDK-process response from
callCurl
In register_hooks
Hooked pre_config
Hooked post_config
Hooked insert_filter
Hooked insert_error_filter
Hooked fixups
Hooked post_read_request
And for webgate, I see:
Registering hooks for apache2entry_web_gate.cpp
Hooked post_config
Hooked handler
Hooked check_user_id
Hooked auth_checker
I thought that the handler functions are called almost last part of the
processing (content generation), and my code is hooked to insert_filter, which
I thought occurs earlier than content generation, so shouldn't my code get
processed BEFORE Apache attempts to process the webgate functions?
How can I get my code to process before the webgate does?
insert_filter is run between the fixups and the handler hooks.
Try to identify who is producing the variables that you need, in which
phase they are available at the earliest. Then identify which part of
web_gate hijacks the processing such that your code is not executed
anymore. I suppose it is one of web_gate's auth_checker or
check_user_id. If it was the web_gate handler then your code would have
run before.
Sorin
Hi Sorin,
I posted a later msg that I've been trying do something along the lines that
you said:
I've been doing more testing, and it appears that the insert_filter hook (the
ap_headers_insert_output_filter() function) is the only place where I can put
my code where it has access to the variables that it needs to do the processing
that I'm doing.
The problem is that if that other Oracle module is enabled in the Apache, it
runs before my code, and I can't get the insert_filter hook (my function) to get
processed before the Oracle module
The SSL variables are set in the fixups hook by mod_ssl. The fixups hook
is run _after_ check_user_id and auth_checker. So you cannot rely on
mod_ssl to populate the environment with the variables. I guess you'll
have to get those variables yourself, before Oracle's check_user_id and
auth_checker hooks.
Sorin
Re: Confused about modules processing order...
On 2012-06-26 19:56, oh...@cox.net wrote: You cannot wait until mod_ssl runs its fixups, you have to hook one of the hooks that execute earlier than webgate's check_user_id or auth_checker. (You have to hook one of the hooks (1)-(4).) There, in your hook, you have to get yourself the values of the server certificates, client certificate, etc, everything that mod_ssl would have given you, but too late. I guess that what I'm seeing is exactly what you said would happen, i.e., my check_user_id hook function is being called, but none of the SSL vars are populated (since, as you said mod_ssl doesn't populate them until the fixup phase). What mechanisms/methods could I use to get those SSL vars (you have to get yourself the values of the server certificates, client certificate, etc, ) at this point? I don't know, unfortunately. Have a look at the sources (modules/ssl/ssl_engine_kernel.c, ssl_hook_Fixup) to see how mod_ssl does it. Apparently mod_ssl uses ssl_var_lookup defined in ssl_engine_vars.c. Maybe you can use it in check_user_id already. Sorin
Re: ssl_var_lookup snippet was Re: Confused about modules processing order...
On 2012-06-26 22:17, oh...@cox.net wrote:
Sorin Manolachesor...@gmail.com wrote:
On 2012-06-26 19:56, oh...@cox.net wrote:
You cannot wait until mod_ssl runs its fixups, you have to hook one of
the hooks that execute earlier than webgate's check_user_id or
auth_checker. (You have to hook one of the hooks (1)-(4).) There, in
your hook, you have to get yourself the values of the server
certificates, client certificate, etc, everything that mod_ssl would
have given you, but too late.
I guess that what I'm seeing is exactly what you said would happen, i.e., my
check_user_id hook function is being called, but none of the SSL vars are
populated (since, as you said mod_ssl doesn't populate them until the fixup
phase).
What mechanisms/methods could I use to get those SSL vars (you have to get yourself
the values of the server certificates, client certificate, etc, ) at this point?
I don't know, unfortunately. Have a look at the sources
(modules/ssl/ssl_engine_kernel.c, ssl_hook_Fixup) to see how mod_ssl
does it.
Apparently mod_ssl uses ssl_var_lookup defined in ssl_engine_vars.c.
Maybe you can use it in check_user_id already.
Sorin
Sorin,
THANKS for that pointer to ssl_var_lookup.
As a very small payback (VERY small) for your help (and others), and for the record, I
put the following code (assembled from various places) in the ap_headers_early, and it
seems to work somewhat)
static apr_status_t ap_headers_early(request_rec *r)
{
printf(In ap_headers_early\n);
printf(\n\nIn ap_headers_early: About to call ssl_var_lookup\n);
typedef char* (*ssl_var_lookup_t)(apr_pool_t*, server_rec*, conn_rec*,
request_rec*, char*);
ssl_var_lookup_t ssl_var_lookup = 0;
ssl_var_lookup = (ssl_var_lookup_t)apr_dynamic_fn_retrieve(ssl_var_lookup);
const char * foo = ssl_var_lookup(r-pool, r-server, r-connection, r,
SSL_CLIENT_CERT);
printf(In ap_headers_early: SSL_CLIENT_CERT=[%s]\n, foo);
.
.
and it seems to work perfectly!!
Do you think that such calls would work in ANY hook? In other words, would I
be at my leisure to use that in ANY of the module hooks?
No, it won't work in any hook, in my opinion. The availability of the
data depends on the phase (hook) in which you run the ssl_var_lookup.
I think, though I'm not sure, that the data are gathered in the
post_read_request hook. If so, ssl_var_lookup would work in any hook
that is called after post_read_request.
ap_headers_early is run in post_read_request. My intuition is that
putting your code there is slightly too early. This is because the
directory-wide configuration of the request is not yet correctly set in
this phase and URL rewrite rules have not yet been applied, although I
don't know if this would affect your functionality.
I'd put the code either in header_parser or in check_user_id and I'd try
to make sure that my check_user_id is run before webgate's check_user_id.
I'd go for header_parser as it is always run for main requests.
check_user_id is run only when some conditions are satisfied (check the
ap_process_request_internal in server/request.c).
If you go for check_user_id, make sure that it is run before Oracle's
check_user_id. In order to do so, you can use APR_HOOK_FIRST
(ap_hook_check_user_id(my_check_user_id, NULL, NULL, APR_HOOK_FIRST)),
or you can use something like
static const char *successor[] = {nameoftheoraclesourcefile, NULL};
ap_hook_check_user_id(my_check_user_id, NULL, successor, APR_HOOK_MIDDLE);
(See how mod_ssl places its post_read_request _after_ mod_setenvif's in
modules/ssl/mod_ssl.c)
Also, I would not change mod_headers, I would write my own module in
which I'd place my header_parser hook.
Sorin
Re: Anyone have some example code doing simple HTTP GET request from within a module?
On 2012-06-23 04:47, oh...@cox.net wrote: Hi, Per earlier threads on this list, I've been working on an Apache module. For the time being, I'm kind of stuck because of the problems that I've run into with trying to integrate my module with a 3rd party library, so just for my module, which is mainly a proof-of-concept, I'd like to have my module do an HTTP GET request. So, I was wondering if anyone has some simple example code for doing that from within a module, maybe using libcurl, or just natively using sockets? I'm trying to do this myself, and I've been looking at using libcurl, but most of the examples that I've seen use the easy setup, so if someone has something like that that can be shared, it'd be a big help. Conversely, if I figure it out, I'll post some working snippets here :)... I'll say the same thing as Ben, try with apache, either mod_proxy or ap_run_sub_request. That if you make one outgoing request per incoming request. If you want several outgoing requests, in parallel preferably, per incoming request, then go with some 3rd-party library. I have some in-house C++ wrappers for libcurl (curl_multi_* + timeouts + client pools), but they are not straightforward to use, a lot of setup is involved, and they are not thoroughly tested. S
Re: Followup to earlier thread about How to compiling/link/use Apache module that uses shared library?
On 2012-06-22 17:35, oh...@cox.net wrote: Sorry. I meant to say: So, my code calls ObConfig_initialize() then it appears that that calls ObConfig::initialize() which is presumably a C++ function. We develop our apache modules in C++ on a regular basis and they interact with other modules written in plain C and there's no problem. What I think happens in your case is: I suspect that the Oracle lib was _statically_ linked with libcrypto. So the code of some version of libcrypto is in the libobaccess binary. Then mod_ssl is _dynamically_ linked with libcrypto. I suspect that the two libcryptos have different versions and they are possibly incompatible = segfaults at all kind of mallocs/frees. I think it has nothing to do with new/delete vs malloc/free. S
Re: How to compiling/link/use Apache module that uses shared library?
On 2012-06-21 19:47, oh...@cox.net wrote: I've tried using -l pointing directly to the .so, libobaccess.so, but when I do that, it says it can't find the .so: [root@apachemodule build-mod_headers]# ./compile-mod-headers.sh /apps/httpd/build/libtool --silent --mode=compile gcc -prefer-pic -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -g -O2 -pthread -I/apps/httpd/include -I/apps/httpd/include -I/apps/httpd/include -c -o mod_headers.lo mod_headers.c touch mod_headers.slo /apps/httpd/build/libtool --silent --mode=link gcc -o mod_headers.la -l/apps/netpoint/AccessServerSDK/oblix/lib/libobaccess.so -rpath /apps/httpd/modules -module -avoid-versionmod_headers.lo /usr/bin/ld: cannot find -l/apps/netpoint/AccessServerSDK/oblix/lib/libobaccess.so collect2: ld returned 1 exit status apxs:Error: Command failed with rc=65536 Try -lobaccess S
Re: How to compiling/link/use Apache module that uses shared library?
On 2012-06-21 22:04, oh...@cox.net wrote: Ben Noordhuis i...@bnoordhuis.nl wrote: On Thu, Jun 21, 2012 at 8:43 PM, oh...@cox.net wrote: I tried that, which allowed me to start Apache, but am getting a segfault. Run it through gdb and inspect the backtrace. Compiling with debug symbols and optimizations disabled (-g -O0) will help. Sorin, The apxs already has -g and -O2 looks like. How do I change that to -O0? Or, do I just run the gcc compile manually? Try adding -Wc,-O0 -Wc,-g -Wc,-fno-inline -Wl,-g to your apxs command line. Also how do I run it through gdb, since apachectl is a script? You don't run apachectl, you run the apache binary. gdb Then in gdb type: file /path/to/your/httpd_or_apache2 set args -d /path/to/your/server_root_dir -f /path/to/your/conf/file -X If apachectl sets some environment variables first, which are then used in the conf file (as in the standard debian installation) then you'll have to set them manually in gdb, e.g. set environment APACHE_RUN_USER www-data set environment APACHE_RUN_GROUP www-data Last, type run For example, on my debian I'd do file /usr/sbin/apache2 set args -d /etc/apache2 -f /etc/apache2/apache2.conf -X And a couple of set environments and then run. S Sorry for the questions, but not too familiar with this stuff :(... Jim
Re: How to compiling/link/use Apache module that uses shared library?
On 2012-06-21 22:22, oh...@cox.net wrote:
[root@apachemodule bin]# gdb httpd
GNU gdb Red Hat Linux (6.3.0.0-1.162.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type show copying to see the conditions.
There is absolutely no warranty for GDB. Type show warranty for details.
This GDB was configured as x86_64-redhat-linux-gnu...Using host libthread_db library
/lib64/tls/libthread_db.so.1.
(gdb) b header_post_config
Function header_post_config not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (header_post_config) pending.
(gdb) run -X
Starting program: /apps/httpd/bin/httpd -X
[Thread debugging using libthread_db enabled]
[New Thread 182897612000 (LWP 8741)]
Breakpoint 2 at 0x2a97a69060: file mod_headers.c, line 1026.
Pending breakpoint header_post_config resolved
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897612000 (LWP 8741)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1026
1026printf(In header_post_config\n);
(gdb) s
1025{
(gdb) s
1026printf(In header_post_config\n);
(gdb) n
In header_post_config
1027header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1029}
(gdb) n
0x004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
(gdb)
I have no idea what the above means :( But it looks like something blew up
in libcrypto?
Jim
Navigate the help of gdb (help, help data, help running).
Check the bt (backtrace), up, and down commands. From your
segfault point in libcrypto you can go up the stack until you reach in
one of your functions.
When you reached your code after several ups, do a print of the
variables (especially pointers that you suspect they are null but
shouldn't be null) that could have triggered the error.
S
Re: How to compiling/link/use Apache module that uses shared library?
And I forgot to say: run gdb in some sort of environment where you see your current source code line and a couple of surrounding lines. You could achieve this with the list command, but I prefer running gdb in emacs and let emacs do the nice listing of source code in a different panel. S
Re: How to access client certificate PEM and incoming request headers in a module?
On 2012-06-18 08:53, oh...@cox.net wrote: Hi, I'll look at ssl_var_lookup a little later, but I'm still messing around with mod_headers.c, tweaking it to understand how THAT is working :)... I added a call to header_request_env_var(r, REMOTE_URI), just to see what it got (running Apache in single-process mode): printf(REMOTE_URI=[%s]\n, header_request_env_var(r, REMOTE_URI) ); Then I pointed a browser to http://myhost/test, where /test was a Location with a RequestHeader (to trigger mod_headers) but I got: REMOTE_URI=[(null)] Shouldn't that be showing: REMOTE_URI=[/test] How did you set the REMOTE_URI environment variable? I've grepped the apache sources and there's no occurrence of REMOTE_URI, so I assume that apache does not set it as part of its request processing. S
Re: Protocol converter module
On 2012-06-05 13:47, Robert Mitschke wrote:
How do I go about implementing this select. I have searched through the
code but could not find out a way to actually get a handle to the socket.
In the code all that is handled are network buckets. How would I gain
access to the socket handle in process_connection? I would need that handle
to select on it.
Place a callback on the pre_connection hook. The 2nd argument to
pre_connection is an apr_socket_t.
int
pre_connection_callback(conn_rec *c, void *sock) {
apr_os_sock_t os_fd;
apr_os_sock_get(os_fd, (apr_socket_t *)sock);
// store os_fd somehwere (for example in your c-conn_config)
// os_fd is your socket descriptor
return OK;
}
Re: Windows service startup configuration - how to find out the working process?
On 04/26/12 00:03, Waldemar Klein wrote: Hello. Looking at the startup procedure of Apache on Windows (as service), I noticed that the httpd.conf is read and processed (i.e. procedures ran) in total 4 times. After startup, only one of them is actually used later to handle requests. I have a module which has a rather expansive (reading and processing a few big files into memory) operation on startup. It is a waste of resources to do this 4 times, when it is used only once. What I found out so far: - the first process (pid A), runs as the user who entered the command httpd -k (re)start. Processes httpd.conf once. This will be terminated after startup is complete. It would not use and memory later on, but still it needs the processing time at startup. - the second (pid B) (runs as SYSTEM) seems to be the control process and only starts another child process (pid B), processes httpd.conf once. Writes its pid to logs/http.pid. Stays as process but doesn't answer requests. - the third (pid C), processes the http.conf twice, only the second one is actually used. I can think of a few quite ugly hacks to solve this: - writing the pids to files and only doing the configuration work when I already see my own pid was in the file, this must be the 2nd run on the process with pid C (can also clean up then). Problemmatic with crashes during startup. - only prepare the config (store filenames) and finish (read and process files) the first time I actually get to work. Problem is, first request might take some time, because the work must be done. Could be solved by a dummy request right after startup. (it's still an ugly hack :) ) - since I can identify the first process (different user), I could start a counter, and only read my files when the counter reaches 4. Problem: this doesnt work if apache is started with -X. (Also I haven't checked yet what the user of the 1st process is if it's started at boot or via a scheduled task) I can think of a few more problems that might arise, and I am even more afraid of the problems that I can not think of right now :) Now my question is, is there a good method to find out if we (the configuration procedures) are in the working configuration? This would be the third process and the 2nd processing of the httpd.conf, which will be actually used when the module gets to work. Try not to do the heavy lifting during the configuration parsing. Put the heavy stuff in the post_config hook. You can combine it with a boolean variable whose state you switch after you did the heavy lifting once. I think it doesn't matter when you parse the conf. If you do it early, the child processes inherit the address space of the parent in which you already did you processing. The differences between the various parsings are the process permissions and availability of the log files. Please note that I didn't work with apache on Windows, so my advice is based on not-so-educated extrapolations. Sorin
Re: Calling another URL from output filter
On 2012-03-04 19:19, Swaminathan Bhaskar wrote: Hello, How can I call another url from an output filter - here is the scenario: when a client accesses abc.mydomain.com/signon, this url autheticates the user and we dont want the response going back to the client rather call another url xyz.mydomain.com/fetchdata which will return some data ... we want to send the data from the second url and response headers from first url merged back to the client. So my thought is to intercept the response from the first url in an output ffilter and make a call from the output filter to the second url. What function call would allow me to make th call to the second url Any help appreciated Hello, We did something similar but we didn't issue the 2nd request from the output filter of the first. The first request was made by the apache subrequest API (the ap_sub_req_lookup_uri and ap_run_sub_req functions). The output filter ran until it encountered an end-of-stream. It did not make any other request. The output filter of the subrequest did not pass any brigade to downstream filters. It simply parsed the response, stored relevant data in some structure and returned APR_SUCCESS to its upstream filters. Next, after ap_run_sub_req returned, we invoked the 2nd URL via the proxy module. The useful data returned by the 1st URL was taken from the structure in which the subrequest stored it. Calling a 2nd URL from output filters is a bit tricky, as you have filter-chains invoked from within a filter-chain, so we preferred to call the 2nd URL only after the request to the first completed. Regards, Sorin
Re: thread ID
On 03/02/12 00:21, Ben Noordhuis wrote: On Thu, Mar 1, 2012 at 17:29,sorin.manola...@orange.com wrote: Hello, I would need a memory buffer associated per worker thread (in the worker MPM) or to each process (in the prefork MPM). In order to do that, I would need a map thread-buffer. So, I would need a sort of thread ID/key/handle that stays the same during the lifetime of the thread and no two threads in the same process can have the same ID/key/handle. What is the most portable way to get this thread ID? I thought of r-connection-id. It works but it is not very portable as it is not guaranteed that two connections created by the same thread will have the same id. They do for now. If r-connection-sbh was not opaque it would be great, because sbh-thread_num would be exactly what I need. I could also use pthread_self. It works too but, in general, it is not guaranteed that the worker threads are pthreads. Thank you for your help. Sorin What about apr_os_thread_current()? It returns a opaque value that's a pthread_t on Unices and a pseudo-HANDLE on Windows. Read this[1] to understand what that means. As a recovering standards lawyer I should probably point out that pthread_t is an opaque type that's not guaranteed to be convertible to a numeric value (or to anything, really). That said, I've never seen a pthreads implementation where that wasn't the case. [1] http://msdn.microsoft.com/en-us/library/windows/desktop/ms683182%28v=vs.85%29.aspx Thank you, it's what I need. Sorin
Re: about setting r-headers_out
On 02/29/12 07:52, Rui Hu wrote: hi, I want to set Content-Type and Cache-Control fields in my private module. So I hooked fixups and used apr_table_setn to set r-headers_out but nothing happened. Was it thought through? Thanks for you help! Try to set r-err_headers_out. For content-type you could check the configuration directive DefaultType. You could also set Cache-Control with the Headers directive. Check its always option too. You can also combine the Headers directive with environment variables set in r-subprocess_env. Check http://httpd.apache.org/docs/2.0/mod/mod_headers.html#header Regards, Sorin
Re: Threads and signals in a module
On 2012-02-28 22:05, Ben Rockefeller wrote: Hello, I have a module which creates a thread using pthread_create and then registers for SIGRTMIN+4 signal from another process. Problem is I do not see a signal callback when the signal is sent to me. I only see it when I try to kill apache where just before dying it hits the signal callback. So something in Apache is blocking the signal. I am using worker MPM. What could be the issue. I do not have much leeway into changing the design of the module (the threading model comes from a separate lib which I link into the module)...but I can change to use a different signal than SIGRTMIN+4, etc. I can also change apache and recompile apache if needed. Please let me know if you have any suggestions. I have been stuck at this for a while now :-( Thanks In which hook do you create your thread and you register your signal handler? Right after executing the child_init callbacks, apache blocks the threads from receiving most signals. Check the list of blocked/ignored/caught signals using ps axs in a shell. Sorin
Re: mod_proxy retry
On Fri, Nov 4, 2011 at 01:18, Jodi Bosa jodib...@gmail.com wrote:
thanks but unfortunately it seems mod_include #include virtual does not
appear to support requests to external servers (must not contain a scheme
or hostname - only path and query string).
Regardless, I also tried ap_sub_req_lookup_uri():
static apr_status_t x_response_filter(ap_filter_t *f, apr_bucket_brigade
*bb)
{
request_rec *r, *subr;
int
status;
request_rec *subr = ap_sub_req_lookup_uri(http://www.apache.org;,
r, f-next);
status = ap_run_sub_req(subr);
ap_destroy_sub_req(subr);
}
which seemed to succeed to make_sub_request() but later on mod_proxy's
proxy_handler() failed because of r-proxyreq was NULL.
You can still do it with ap_sub_req_lookup_uri. Just use something like
RewriteEngine On
RewriteRule /dummy/path http://external.proxy/path2 [P]
and then ap_sub_req_method_uri(GET, /dummy/path, r, f-next);
The [P] in the rewriterule makes sure you have r-proxyreq not null.
S
Re: Developing Authn/Authz Modules
On Sat, Oct 1, 2011 at 23:05, Suneet Shah suneetshah2...@gmail.com wrote:
Hello,
I am trying to build my apache module which needs to carry out
authentication and authorization functions based on the value of a cookie.
To start with, I have just created a shell with the intent that I wanted the
functions for authentication and authorization being called.
However, it does not appear that these functions are being called. I have
pasted by configuration and code below.
When I try to access http://localhost/test_rpc/ I get the login.html that
is defined in my ErrorDocument below.
But when I look in the log file, I see the following.
Since its looking for a userId, I am wondering if there is an error in my
configuration
[Sat Oct 01 16:37:29 2011] [debug] prefork.c(996): AcceptMutex: sysvsem
(default: sysvsem)
[Sat Oct 01 16:38:08 2011] [error] [client 127.0.0.1] access to
/test_rpc/header.jsp failed, reason: verification of user id 'null' not
configured
You have not hooked check_user_id. In this case the default
check_user_id of mod_authn_default is called. The mod_authn_default
module rejects the request by default and gives you the verification
of user id 'null' log line.
Hook check_user_id instead of auth_checker. Set r-user in
check_user_id. I think setting r-user is not mandatory but it gives
you more precise log messages.
Use return OK (OK is 0) and not return HTTP_OK (HTTP_OK is 200) in your hooks.
S
Any guidance on what I am doing wrong would be greatly appreciate.
Regards
Suneet
-- Configuration in Httpd.conf
Location /
IAM_CookieName IAM_PARAM
IAM_TokenParam tkn
IAM_Service_base_url http://localhost:8080/;
ErrorDocument 401 /login.html
AuthType IAMToken
AuthName IAM Login
AuthCookie_Authoritative On
/Location
Location /test_rpc/
ProxyPass http://localhost:9080/test_rpc
require tkn
/Location
- Module Code
static int authz_dbd_check(request_rec *r) {
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server, authz_dbd_check
called);
return HTTP_OK;
}
static int check_token(request_rec *r) {
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server, chedk_token
called.);
return OK;
}
static void authz_dbd_hooks(apr_pool_t *p)
{
ap_hook_auth_checker(check_token, NULL, NULL, APR_HOOK_MIDDLE);
ap_hook_auth_checker(authz_dbd_check, NULL, NULL, APR_HOOK_MIDDLE);
}
module AP_MODULE_DECLARE_DATA authz_dbd_module =
{
STANDARD20_MODULE_STUFF,
authz_dbd_cr_cfg,
NULL,
NULL,
NULL,
authz_dbd_cmds,
authz_dbd_hooks
};
Re: Question on sub requests and output filter context.
On Thu, Sep 15, 2011 at 12:52, Martin Townsend martin.towns...@power-oasis.com wrote: Hi, I have an output filter that parses custom tags to retrieve data from an application running on the same device. Everything was working well until I tried to move some HTML into Server Side Include pages. Snippet below: ?smu smu extio_sensor_read mappings ? ?smu smu extio_read front_ana all led ? ?smu smu extio_read rear_ana all led ? !--#include virtual=/include/SSI_SensorStatus.html -- !--#include virtual=/include/SSI_SensorStatusAnalogRear.html -- The first three commands will populate hash tables that are saved in my output filters context. The HTML in the included pages then use custom tags to query the hash tables but for some reason the hash tables are NULL. Having stepped through with the debugger I can see that the pointer to the output filter when processing the main HTML page is different to the one when parsing custom tags in SSI pages. Looking through mod_include I can see it creates a sub request for include and sub requests call make_sub_request to create a new filter. Should this new filter also inherit the output filters context? Am I doing something wrong with my use of mod_include? I've tried moving my filter so it's after mod_include but still the same problem. I'm using Server version: Apache/2.2.19 (Unix) on an ARM board. Best Regards, Martin. How do you construct the context of your filter? At the first invokation of the filter or in the init function of the filter? In the second case, it could be that you construct the context twice, the first time in the main request processing and the second time in the subrequest processing. In my opinion, apache uses the same filter structure in both the main and the sub request. In mod_includes apache creates a subrequest, passing f-next to it. Thus, the first filter in the filter chain of the subrequest is the filter succeeding the INCLUDES filter. In my opinion, if you place your filter before the INCLUDES filter, your filter should not be called in the subrequest if yours is a AP_FTYPE_RESOURCE filter. If you place your filter after the INCLUDES filter, the hash tables you mention are not initialised at the time when your filter processes the responses of the includes subrequests. I am not sure of what I'm saying because I have no experience in how mod_includes interacts with other filters. Anyway, I hope this helps. Have a look in server/request.c at make_sub_request. The subrequest inherits the protocol filters of the main request, but not all of the non-protocol output filters of the main request. Maybe you should make your filter a AP_FTYPE_PROTOCOL filter such that it is not removed from the chain by mod_includes. S
Re: My first Apache module!
On Wed, Aug 24, 2011 at 23:59, Chris London ch...@kwista.com wrote: Hey everyone, I don't like to bother mailing lists with beginner questions so I spent a few hours on Google looking for info and tutorials. I'm sure at least one of you has had that experience :) Anyway, I have finished my first module and it works exactly how I want it to. I still have more features I want to add but I'm a little rusty with my C and even worse at Apache Modules. Would you mind taking a glance at my code to see if I'm doing anything wrong or have any memory leaks or anything? Here's the source code: https://github.com/chrislondon/Dynamic-Stylesheets/blob/master/mod_dss.c Feel free to fork my project: https://github.com/chrislondon/Dynamic-Stylesheets The README file ( https://github.com/chrislondon/Dynamic-Stylesheets/blob/master/README) contains usage info and the project contains a simple test case. Thanks everyone! You open the file but then you exit the handler if r-header_only without closing the file. Check for r-header_only before opening the file. In T_read you allocate cKey, cVal, and temp_node once for every iteration of the outermost while loop in which *cCurrent == '@' , but you free this memory only once, outside the while loop. Also, you do not free correctly your linked list. S
Re: ? symbol in the URL for ProxyPass
On Fri, Aug 19, 2011 at 15:29, Denys dhryvas...@serena.com wrote: Hello - I am trying to use Apache Server as a proxy server for my applications. I try to make the following mappings: ProxyPass /test/pass1/ http://localhost:80/denys/denys.dll? ProxyPass /test/pass2/ http://localhost:8085/violetta/ I have two apps running under Jboss and IIS. The second mapping is working perfectly. The first one shows me 404 error. Does anybody know, is it because of ? symbol at the end of the URL? If I try any other URL - it works fine. Are there any solutions for this case? Yes, it is because of the '?'. If you want to append query string arguments, use RewriteRules. S Any help is greatly appreciated. Thanks in advance, Denys -- View this message in context: http://old.nabble.com/%22-%22-symbol-in-the-URL-for-ProxyPass-tp32295250p32295250.html Sent from the Apache HTTP Server - Module Writers mailing list archive at Nabble.com.
Re: How to test if request has been aborted
On Sat, Jul 30, 2011 at 01:02, Tony Abo t...@hitech.com wrote:
On Fri, Jul 29, 2011 at 02:35, Tony Abo t...@hitech.com wrote:
I need to cut the processing short if the user decides to press
the
stop button on the browser. I cant seem to figure out how to test
for
that condition from inside the handler. Can anyone help me?
Thanks in advance,
Tony
r-connection-aborted
Cheers
Tom
Thanks Tom
Will that value get updated asynchronously if the connection closes
while my handler does its processing (I.e without calling any Apache
functions)?
My testing shows that connection-aborted is not being set
asynchronously when the connection is closed by the client. I need one
of the following:
- Some Apache function I can call that will attempt to touch the open
socket and either set connection-aborted or return an error status so I
can know it is no longer connected.
Or
- Access to the actual socket buried somewhere in the connection
structure. I can't seem to find it. If I had that, I could test it
myself.
The earliest hook that is passed the socket is create_connection. The
socket is passed in the third argument. Use apr_os_sock_get to get the
OS-specific socket descriptor.
If you do not place your own callback on the create_connection hook in
order to save the socket in your own structures, then you can use the
method below, but it's a hack, as I guess the core_module structure is
not supposed to be visible to modules. The method works after the
pre_connection hook.
#define CORE_PRIVATE 1
#include http_core.h
apr_socket_t *sock = (apr_socket_t
*)ap_get_module_config(r-connection-conn_config, core_module);
apr_os_sock_t fd; // int for Unix
apr_os_sock_get(fd, sock);
#undef CORE_PRIVATE
Thanks Sorin, that works. I found another approach, but I'm not sure how safe
it is.
int CheckConnected(request_rec r)
{
int nSocket;
char acBuffer[1];
int nResult;
core_net_rec *cnr =
pConnID-pRequestRecord-connection-output_filters-ctx;
output_filters is a linked list of filters. I think that it cannot be
guaranteed that the first filter in the linked list is the core output
filter. Thus, one cannot guarantee that output_filters-ctx exists and
that it is a core_net_rec.
if ( apr_os_sock_get( nSocket, cnr-client_socket ) != APR_SUCCESS )
return FALSE;
nResult = recv( nSocket, acBuffer, 1, MSG_PEEK | MSG_DONTWAIT );
return nResult 0 || (nResult == -1 errno == EWOULDBLOCK);
}
Any thoughts?
Thanks again,
Tony
Re: How to test if request has been aborted
On Fri, Jul 29, 2011 at 02:35, Tony Abo t...@hitech.com wrote: I need to cut the processing short if the user decides to press the stop button on the browser. I cant seem to figure out how to test for that condition from inside the handler. Can anyone help me? Thanks in advance, Tony r-connection-aborted Cheers Tom Thanks Tom Will that value get updated asynchronously if the connection closes while my handler does its processing (I.e without calling any Apache functions)? My testing shows that connection-aborted is not being set asynchronously when the connection is closed by the client. I need one of the following: - Some Apache function I can call that will attempt to touch the open socket and either set connection-aborted or return an error status so I can know it is no longer connected. Or - Access to the actual socket buried somewhere in the connection structure. I can't seem to find it. If I had that, I could test it myself. The earliest hook that is passed the socket is create_connection. The socket is passed in the third argument. Use apr_os_sock_get to get the OS-specific socket descriptor. If you do not place your own callback on the create_connection hook in order to save the socket in your own structures, then you can use the method below, but it's a hack, as I guess the core_module structure is not supposed to be visible to modules. The method works after the pre_connection hook. #define CORE_PRIVATE 1 #include http_core.h apr_socket_t *sock = (apr_socket_t *)ap_get_module_config(r-connection-conn_config, core_module); apr_os_sock_t fd; // int for Unix apr_os_sock_get(fd, sock); #undef CORE_PRIVATE Thanks again, Tony
Re: Finding out original request's scheme
On Thu, Jul 21, 2011 at 11:02, Ignaz Birnstingl ign...@gmail.com wrote: Hello, is there an easy way to find out a request's scheme part (i.e. http or https)? r-parsed_uri.scheme seems to be a null pointer. I currently iterate r-input_filters and check if filter-frec-name is ssl/tls filter but that doesn't seem right. Any suggestions? Check the HTTPS environment variable (r-subprocess_env). S
Re: Sharing information between threads and processes.
On Thu, Jul 21, 2011 at 13:25, Zaid Amireh tum...@gmail.com wrote: On Jul 21, 2011, at 1:53 PM, Nick Kew wrote: Indirection in shared memory is inherently complex! How near do the socache modules come to meeting your needs? mod_disk_cache would unfortunately make my code pretty complex and maybe slow as I'm not caching documents but rather tokens and strings. mod_mem_cache is a per-process cache which simply doesn't meet the requirements. I'm open to any wild ideas :) C++, boost::interprocess, STL-like structures in shared memory? S
proposed extension to mod_headers
Hello, Some while ago I've posted a messages in which I enquired if there is a module that copies a slightly modified output header to a response. In particular, I wanted to set the same cookie on two domains in the same request. The content generator sets the cookie on one domain and I wanted to copy the Set-Cookie header and to modify its domain field. For example: The content generator sets: Set-Cookie: name=value; domain=.sub.domain.net; expires=12000 and I want that the client gets: Set-Cookie: name=value; domain=.sub.domain.net; expires=12000 Set-Cookie: name=value; domain=.domain.net; expires=12000 I've developed a module that is similar to mod_headers. I've added a directive called HeaderExt that would be used as shown below in order to obtain the functionality shown in the example above. HeaderExt [always|onsuccess] copyedit Set-Cookie ^name=(.*)domain=[^;]+(.*) name=$1domain=.domain.net$2 [env=var] The code is available here: http://code.google.com/p/thought-tracker/source/browse/#hg%2Fcpp%2Fmod_headers2 Thank you for your feedback. Sorin
Re: input filters called again after Handler returns
On Thu, Jun 30, 2011 at 02:56, Jodi Bosa jodib...@gmail.com wrote: I'm encountering a strange interaction between modules (including my own). When I track it down, it appears that input filters are called after the handler is finished which results in 2 bodies in the response. In other words, this is what appears to be happening: Input filters called with AP_MODE_GETLINE (for each HTTP request header) Handler called and returns 302 input filters called with AP_MODE_READBYTES and 8192 I've placed logging in ap_internal_*_redirect() and removed all related modules to see if one was a problem - yet the same issue happens. Thanks for any help. At the end of the request processing chain, apache places a function that clears the request body. (ap_finalize_request_protocol-ap_discard_request_body). Your inut filter should place an EOS bucket in the brigade or return something else than APR_SUCCESS in this second invocation. Sorin
Re: Module External Configuration
On Tue, Jun 21, 2011 at 23:26, Jason Funk jasonlf...@gmail.com wrote: One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? Destroying the shared memory segment with the conf could be dangerous as other children could serve requests that are using the conf in the segment that you are destroying. Protecting with interprocess locks significantly slows down your server. Maybe a shared-exclusive lock is more efficient, I don't know. You could use a double buffer approach and reference counters. For example you load the new conf in buffer 2, then you update the pointer that points to the active conf, setting it to buffer 2. Then you check the reference counter of buffer 1 and when it reaches zero, you destroy buffer 1. The pointer that points to the active buffer, as well as the reference counters have to be mapped in shared memory too. S Jason On Mon, Jun 20, 2011 at 5:20 PM, Nick Kew n...@apache.org wrote: On Mon, 20 Jun 2011 16:10:12 -0400 Mike Meyer m...@mired.org wrote: I assume that this is because a new process was spawned to handle a new request and the updated memory didn't get carried over (even though the pointer address didn't change...) A new process may be spawned from time to time (though not directly to handle a new request unless you have a very non-standard MPM). You could check config data in a child_init hook. However, if you have this problem, it's probably not your only one. The data reverting suggests that your module's config may be held on the server/config pool. If that's being updated (and if the updates involve any allocation at all) it's a memory leak. Your module should create its own config pool at child_init. Then it can read its own config data immediately, and - crucially - whenever it re-reads the data it can first clean that pool to reclaim memory. Put the configuration data into shared memory. Create and load the shared memory in the post config hook. Map the shared memory and potentially reload it during the child init hook. You'll need to use appropriate locking if you decide to reload it. Details on the locking will depend on the data structure details. If you use shared memory, note that 2.3/2.4 have much better support for it than earlier versions, with mod_socache and mod_slotmem. However, this may not be the best thing to do. You need to weigh up the cost of maintaining a per-process copy of the config data against that of using shared memory. Both are valid solutions. Also, do you want the overhead of a stat() every request to see if anything has changed, or could it work on a time basis, so it only performs a stat() if a certain time has elapsed since the last time? -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: adding and editing response headers in conf
On Thu, Jun 16, 2011 at 18:41, Joe Lewis jle...@silverhawk.net wrote: On Thu, 2011-06-16 at 12:32 -0400, Shawn Ligocki wrote: On Thu, Jun 16, 2011 at 11:57 AM, Joe Lewis jle...@silverhawk.net wrote: On Thu, 2011-06-16 at 17:46 +0200, Sorin Manolache wrote: Can I get this response just by changing the configuration of apache? Header edit cookie_name(.*)domain=[^;]+(.*) cookie_name$1domain=.domain.net$2 does not help as it only moves the cookie from one domain to the other and I want it copied, not moved. That is really how it should be. A second header of the same name isn't really allowed in the specification. I believe the HTTP spec does allow multiple Set-Cookie HTTP headers: From RFC 2616, Section 4.2http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.4.2.p.5 : Multiple message-header fields with the same field-name *may* be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It *must* be possible to combine the multiple header fields into one field-name: field-value pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy *must not* change the order of these field values when a message is forwarded. Sorin, there is your answer. Set the header to a single value containing both cookies. Thanks, Shawn! Thanks, I've tried, but it does not work. First, Firefox seems to simply ignore the second cookie. I do Set-Cookie: a=1,b=2 and I see only a=1 in my cookie collection. Second, there's the problem with expires. Typically its syntax is Mon, 15-Aug-2011 10:00:00 GMT, i.e. it contains a comma. So I think that Set-Cookie does not satisfy the condition It *must* be possible to combine the multiple header fields into one field-name: field-value pair, without changing the semantics of the message Sorin
Re: Add a variable on http header
On Mon, Jun 6, 2011 at 14:15, Maurizio Totti maurizio.to...@gmail.com wrote: Hi, I would try to set an http header variable on the fly but without success :-( I write this simple module to test the functionality http://pastebin.com/b8hcZTtb When running your code, I see USER_VAR2: mytest in the response. Be aware that your code adds a _request_ header (r-headers_in) to an _output_ filter. S I don't understand my error, someone can help me? Thanks a lot Regards. -- Maurizio Totti
Re: Add a variable on http header
On Mon, Jun 6, 2011 at 16:14, Maurizio Totti maurizio.to...@gmail.com wrote: 2011/6/6 Simone Caruso i...@simonecaruso.com: Changing,in line 22, from 'headers_in' to 'headers_out' make your module work in my test env only for non-html content. Of course i set SetHandler getadsldap in my conf. Hi at all, @Ben Noordhuis: I had tried all of them to test all the possibility. I don't get any error (in error.log on debug log level) but the USER_VAR2 isn't in my http header. Yes I've found the source code of this module, but my test code it's very more easy, and I've not found what I have lost :-( @Sorin Manolache @Simone Caruso: Ops... :-( It's correct now, but no change in my http response. I think that an AP_FTYPE_RESOURCE filter is invoked _after_ the HTTP response headers have been sent to the client. Try AP_FTYPE_PROTOCOL - 1 instead of AP_FTYPE_RESOURCE. S My backend's test it's done with phpinfo() function. Maybe it's a wrong method? Thanks at all for your help :-) Regards -- Maurizio Totti
Re: I don't understand apr_array_push
In my opinion the problem is here:
alias = (alias_t *)reqc-aliases-elts[k];
Try
alias = ((alias_t *)reqc-aliases-elts)[k];
S
On Wed, Jan 26, 2011 at 16:28, Simone Caruso i...@simonecaruso.com wrote:
Hi list,
i can't understand the reason i can't read from an apr array, my code is
like this:
//The 2 arrays, now working on reqc-aliases
reqc-aliases = (apr_array_header_t *)apr_array_make(r-pool, 5,
sizeof(alias_t));
reqc-redirects = (apr_array_header_t *)apr_array_make(r-pool, 5,
sizeof(alias_t));
while (attributes[i]) {
if (strcasecmp (attributes[i], apacheServerName) ==
0) {
reqc-name = apr_pstrdup (r-pool, vals[i]);
} else if (strcasecmp (attributes[i],
apacheServerAdmin) == 0) {
reqc-admin = apr_pstrdup (r-pool, vals[i]);
} else if (strcasecmp (attributes[i],
apacheDocumentRoot) == 0) {
reqc-docroot = apr_pstrdup (r-pool,
vals[i]);
} else if (strcasecmp (attributes[i],
apacheScriptAlias) == 0) {
cur = strstr(vals[i], );
if(cur - vals[i] 1 ){
tmp = apr_palloc(r-pool,
sizeof(char)*strlen(vals[i]));
strcpy(tmp, vals[i]);
tok = NULL;
alias =
apr_array_push(reqc-aliases);//HERE THE PUSH
alias-src = apr_strtok((char *)tmp ,
, tok);
alias-dst = apr_strtok(NULL, ,
tok);
alias-iscgi = 1;
isalias = 1;
}else{
ap_log_rerror(APLOG_MARK,
APLOG_DEBUG|APLOG_NOERRNO, 0, r,
[mod_vhost_ldap_ng.c]: Wrong
apacheScriptAlias paramter: %s, vals[i]);
}
} else if (strcasecmp (attributes[i], apacheAlias)
== 0) {
cur = strstr(vals[i], );
if(cur - vals[i] 1 ){
tmp = apr_palloc(r-pool,
sizeof(char)*strlen(vals[i]));
strcpy(tmp, vals[i]);
tok = NULL;
alias =
apr_array_push(reqc-aliases); //HERE THE PUSH
alias-src = apr_strtok((char
*)vals[i] , , tok);
alias-dst = apr_strtok(NULL, ,
tok);
alias-iscgi = 0;
isalias = 1;
Now i need to read the array (i read mod_alias.c):
for(k = 0; k reqc-aliases-nelts ; k++){
alias = (alias_t *)reqc-aliases-elts[k];
isalias = alias_matches(r-uri, alias-src);
if(isalias 0)
break;
}
I don't understand why, when k=2, alias structure is filled with wrong
addresses:
(gdb)
658 isalias = alias_matches(r-uri, alias-src);
(gdb)
Program received signal SIGSEGV, Segmentation fault.
0x7fc350e6784f in alias_matches (uri=0x94f820 /,
alias_fakename=0xda0094ff Address 0xda0094ff out of
bounds)
I tried a for like this without success (like mod_alias.c):
for(k = 0; k reqc-aliases-nelts -1 ; ++k){
Any idea?
thanks!
--
Simone Caruso
IT Consultant
+39 349 65 90 805
p.iva: 03045250838
Re: Re: How to send a jpeg-file in Handler
2011/1/19 Whut Jia whut_...@163.com: Hi Sorin Manolache, According your ways(ap_send_fd()),it is work.The picture is returned to client.But myself-set cookie content are not returned , why?? Begin sending jpg-file,I set a cookie in headers_out : r-content_type=image/jpeg; apr_table_setn(r-headers_out,Set-Cookie,cookie-contence); ap_send_fd(fd, request, 0, file_size, bytes_sent); return OK; Besides,the request address of the jpeg picture ishttp://www.whut.com/portal/logout.jpg. This request is sent from the response page of ahttp://www.jjsonicIDP.com/saml/logout.action request . Thanks, ajxs I don't know why the cookie is not sent. Try apr_table_set(r-err_headers_out, ...); Also, are you sure that the pointer to cookie_contents is still valid at the very end of the request processing, that is _after_ you return from the handler. If not, try apr_table_set instead of apr_table_setn. Sorin
Re: Apache log modules
On Mon, Nov 29, 2010 at 00:56, Andrej van der Zee
andrejvander...@gmail.com wrote:
Hi Sorin,
Thanks for your reply.
request_rec-connection-id is a long int that is unique. It is built
from the process_id and thread_id of the apache thread that serves the
request.
Will this be unique for MPM worker across control processes / worker threads?
Yes. It's unique across sites too as it contains the server IP address as well.
However, a client may open several connections to the server
during the same transaction, so I guess this does not help you much.
My assumption is that both client and server have KeepAlive enabled.
In that case, should there generally not be just one connection
only?
As other people remarked, browsers open several connections even if
they and the server support keepalive. Just clear your browser cache
and surf to any site while running netstat in a console.
There's a module called unique_id. It creates a string that is stored
in the req-subprocess_env and can be logged with %{UNIQUE_ID}e. It
encodes the request timestamp, the connection-id, the _server_ IP and
a random number. It does not encode the client IP or port. However, if
you combine it with the client-IP and client port that you can log as
well in the same log line, you could, probably, extract what you want
after some log-postprocessing.
So I can decode the unique ID and extract the connection ID from it? I
guess so...
Different clients behind a NAT router will use different ports.
However, based solely on ports, you won't be able to distinguish
between two different clients on one hand or one client that makes
several connections on the other hand. A method that is used by some
of my colleagues in order to distinguish between different clients
(but I don't know much about it, so I can't tell you more) is to
analyse the TCP header of the packet in order to extract the TCP
sequence numbers.
That sounds interesting but as far as I can see this information is
not available in the application layer. Hard to imagine, but are there
any tricks to get this information in Apache modules? Or are your
colleagues using sniffers such as tcpdump?
Indeed it is not available at the application layer. I do not know how
they do it.
S
Re: Howto unittest httpd modules?
On Thu, Nov 11, 2010 at 08:11, Edgar Frank ef-li...@email.de wrote: Hi modules-dev-folks, I've written a handful of modules for httpd. I'm now looking for a way to setup some unit tests. We have continuous integration running, so I want to supply some tests, starting from compiling against httpd to basic functionality to more elaborate feature tests. I wonder how to unit-test this, as the prerequsites are rather complicated. The tests would have to setup a httpd, provide a config, compile and install the modules. As you don't want to modify the modules themselves, you have to run a bunch of requests and monitor expected output - maybe measuring coverage or running valgrind on the way. I see no way to run real unit tests as you would have to emulate httpd and run your modules against it, as most of the code is httpd or at least highly APR dependent. I see no point in emulating httpd as you would have to implement every little detail - httpd specific behaviour (e.g. in version bumps) is one important thing to test IMHO. So, has anyone some experience with this, some suggestions, howtos or tutorials? Any help would be greatly appreciated. Regards, Edgar In our group we unit test only parts of our modules. We unit test the callbacks and their sequence (a sort of poor-man's emulation of httpd). We populate a request_rec structure and then we pass it to the various callbacks. Our callbacks are mainly wrappers around bigger functions that are httpd-independent. The apache filters are tested in this way. The part that needed more code in order to emulate httpd was the subrequest infrastructure. Sorin
Re: How to init an mmaped file?
On Tue, Oct 26, 2010 at 06:47, Mike Meyer mwm-keyword-apache.b9a...@mired.org wrote: On Tue, 26 Oct 2010 06:03:48 +0200 Ben Noordhuis i...@bnoordhuis.nl wrote: Mike, is your code available anywhere? Unfortunately, a lot of what happens after the connection is made is proprietary. I might be able to pull together enough releasable code to show what's going on up to the point the connection hook is called. Would that help? mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail - www.asciiribbon.org Before invoking the process_connection callback, apache updates the base_server field of the conn_rec structure based on the client IP. The error could be that you mmap the file in one config object but you get another config object as an argument to the process_connection callback. In order to check this hypothesis, log the value of the pointer returned by create_server_config or merge_server_config and the value of the config object pointer that you get in the process_connection invokation. Sorin
Re: How to init an mmaped file?
On Tue, Oct 26, 2010 at 20:36, Mike Meyer mwm-keyword-apache.b9a...@mired.org wrote: On Tue, 26 Oct 2010 10:03:11 +0100 Nick Kew n...@apache.org wrote: On 26 Oct 2010, at 03:52, Mike Meyer wrote: Anyone got some hints on how I ought to set this up? Maybe there's a writeup somewhere I missed on config processing Possibly there's some facility that I ought to be looking at to replace the mmaped file? One possible suggestion. post_config and child_init get passed the first server_rec. The server_rec in any later hook is the virtual host. Could your problem be setting data in one server then trying to retrieve it from another? That indeed appears to be the case (Sorin got this also, pointing out that it comes from using a virtual server). Since I want this to work properly with multiple virtual servers mapping different files, I'd like this to work. To me, the obvious place to put this would be the config merge hook, except that doesn't get passed a copy of the server rec, and the pool it gets passed is apparently different (or possibly reset) between invocations, so the trick of using pool_userdata to skip doing this during the config check doesn't work. I don't see a hook that's obviously convenient for this. Maybe there's one with a non-obvious name? Barring that, it would seem that I have to have both my post_config hook and child_init hook walk through the servers checking their config to see if my module is enabled, and setting them up appropriately. I did something similar. As you say, I had to walk through the servers in both hooks. I can lookup up the code tomorrow if you want. I do not know the pool_userdata trick. What I did was to apr_pool_cleanup_register a callback on the config pool that undid what the post_config callback did. The config pool is cleaned up between the two invocations of the post_config callback. Sorin
Re: add cookie in handler process
2010/9/19 whut_jia whut_...@163.com: Hello, I am new to apache module development,and now I have a problem. Now,I am writing a handler module.In this module,I need validate username/password infomation sent by user.After validating,I set cookie into headers_out(apr_table_set(r-headers_out,Ser-Cookie,)),and then do external redirection (apr_table_setn(r-headers_out,Location, URL)).The question is that I just get Location header but get Cookie header when I access to apache server.why?? Many thanks, Jia Use r-err_headers_out
Re: Problem with ap_internal_redirect_handler
On Wed, Aug 18, 2010 at 14:24, Martin Townsend
martin.towns...@power-oasis.com wrote:
Hi,
I'm currently trying to add the ability to redirect requests for the initial
html page to a captcha page when there is no session or no valid session.
Currently I'm doing this in the fixups hook:
mod_smu_fixups(request_rec* r) {
}
So if we have an initial request that is HTML then redirect to captcha.shtml
which contains SSI for header, footer etc. There will also be a request for
a captcha image using a fixed filename. The fixup will detect this and send
the request to a custom handler that will auto-generate the captcha image.
The problem I have is that I'm getting sub-requests and requests for the
captcha image twice, see log below. I assume that the original request for
index.shtml and the request for captcha.shtml are being processed. Using
firebug I see that the captcha image that is displayed in the debug output
isn't the one that gets displayed in the HTML so I can assume that the image
is being sent twice. I notice that the browser goes into a waiting state
until the timeout expires. Does anybody know what I'm doing wrong? I've
tried returning DECLINED or DONE after the ap_internal_redirect_handler call
but still get the problem. Any help appreciated.
Best Regards,
Martin.
Try putting
if (!ap_is_initial_req(req))
return DECLINED;
at the beginning of your fixups callback.
Sorin
Re: *** glibc detected *** double free or corruption (!prev) in cleanup function
On Thu, Apr 22, 2010 at 05:47, Andrej van der Zee
andrejvander...@gmail.com wrote:
Hi,
I have a question about apr_pool_cleanup_register for a child's pool.
I register a cleanup function that is called when the pool is
destroyed. In the cleanup function, I join a background thread that
first writes some log to a database:
static void mbrace_child_init(apr_pool_t *pool, server_rec *s)
{
apr_pool_cleanup_register(pool, 0, mbrace_bgthread_cleanup,
apr_pool_cleanup_null);
}
static apr_status_t mbrace_bgthread_cleanup(void*)
{
req_log_mngr-stop()
}
But, when it connects to the database in the background thread, it
sometimes ends up in the error below. I do not fully understand why,
but are there any restrictions to what I can do in the cleanup
function?
When running gdb on the generated core-dump, you can see that it
happens on line 134.
125 APR_DECLARE(void) apr_allocator_destroy(apr_allocator_t *allocator)
126 {
127 apr_uint32_t index;
128 apr_memnode_t *node, **ref;
129
130 for (index = 0; index MAX_INDEX; index++) {
131 ref = allocator-free[index];
132 while ((node = *ref) != NULL) {
133 *ref = node-next;
134 free(node); here it happens!!
135 }
136 }
137
138 free(allocator);
139 }
Hopefully somebody can push me in the right direction.
Do you create a background thread per apache process? Or do you have a
single background thread that is used by all apache processes? Which
hook creates the background thread?
It is possible that you create the thread once but you try to join it
several times. I don't know if this is sufficient to get the symptoms
that you see.
Try compiling both apache and apr with debug symbols and run it in a
debugger with -X (i.e. monoprocess). Place a breakpoint on line 134
and see how many times you reach it.
S
Re: Issuing a client side HTTP request from a module
On Wed, Apr 21, 2010 at 22:51, Nick Kew n...@apache.org wrote: On 21 Apr 2010, at 20:42, Some Guy wrote: Hi all, Is there any facility in the Apache libs that allows module developers to act as a client? The usual way would be to harness mod_proxy, commonly in a subrequest. You don't have to do anything special, just request something configured to proxy. As Nick says, the common solution are subrequests. However, note that subrequests are not kept alive. S
Re: LD_PRELOAD for modules
On Sat, Mar 27, 2010 at 12:28, Andrej van der Zee andrejvander...@gmail.com wrote: Hi, Maybe a bit off-topic for this mailing list, but I need to do the same preload-trick for CGI applications written in C++. I was thinking about a wrapper bash-script that usese LD_PRELOAD in addition to a rewrite rule in Apache directive. I have some problems with the wrapper script: #!/bin/bash LD_PRELOAD=/path/to/modified/mysql/libmysql_client.so /mbrace/htbin/cpp-appl.cgi Obviously this does not work because I need to redirect stdin to cpp-appl.cgi and return whatever cpp-appl.cgi through stdout. Don't know how to do this. I am sure I am not the first one to do this, but can't find any useful examples. I was hoping you could help me out? Thank you, Andrej In the Debian package of apache there's /etc/apache2/envvars. I'm not sure if this file appears in an out-of-the-box compilation of apache2. Anyway, /usr/sbin/apache2ctl looks for it and loads it before launching the server. The server in launched in the environment that is set in envvars. So you could try writing export LD_PRELOAD=... in envvars. I do not know if this has an effect on how apache2 spawns the CGI applications but you might give it a try. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: Using aprec in a handler to retrieve request body
2010/2/16 Frédéric Laurendeau frederic.laurend...@gmail.com:
Hi,
I am trying to write a module which will get the request body, process it
and write generated content in the response.
I have seen that some of you used aprec to get the request body but I just
cannot manage to build my module with it. I'm using apxs to build my module
and after the build, there is an error when trying to start apache2 with my
module: Cannot load mod_maquette.so into server: undefined symbol:
apreq_params_as_string
I put at the bottom of the mail my source code and my build script, maybe I
forgot to include some headers in the source or some libraries in the build
command line ?
I don't really understand how all of this works but aprec seems very
powerfull and I'd like to find some tutorials or examples on how to build
modules with it.
Thanks for your help.
Frederic
Build script :
/usr/local/apache2/bin/apxs -c mod_maquette.c
Try
/usr/local/apache2/bin/apxs -c -l apreq2 mod_maquette.c
/usr/local/apache2/bin/apxs -i -a -n maquette mod_maquette.la
Source code mod_maquette.c:
#include stdlib.h
#include math.h
#include string.h
#include httpd.h
#include http_connection.h
#include http_config.h
#include http_core.h
#include http_log.h
#include http_protocol.h
#include ap_config.h
#include http_request.h
#include apr.h
#include apr_strings.h
#include apr_lib.h
#include apreq2/apreq_module.h
#include apreq2/apreq_param.h
static int maquette_handler(request_rec *r)
{
const apr_table_t* args;
apreq_handle_t* h = apreq_handle_apache2(r);
apreq_body(h, args);
const char *params = apreq_params_as_string(r-pool, args, NULL,
APREQ_JOIN_QUOTE);
fprintf(stderr, =TEST=\n%s\n==TEST=\n, params );
ap_rprintf(r, Request data read : %s\n, params);
fflush(stderr);
return DONE;
}
static void register_hooks(apr_pool_t *p)
{
ap_hook_handler(maquette_handler, NULL, NULL, APR_HOOK_MIDDLE);
}
module AP_MODULE_DECLARE_DATA maquette_module =
{
STANDARD20_MODULE_STUFF,
NULL, /* dir config creater */
NULL, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
NULL, /* command apr_table_t */
register_hooks, /* register hooks */
};
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
Re: Using aprec in a handler to retrieve request body
2010/2/16 Frédéric Laurendeau frederic.laurend...@gmail.com:
Thanks for your help !
I tried it but it doesn't work, I get the following errors.
/usr/local/apache2//build/libtool --silent --mode=compile gcc -prefer-pic
-DAP_HAVE_DESIGNATED_INITIALIZER -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -g
-O2 -pthread
-I/usr/local/apache2//include -I/usr/local/apache2//include
-I/usr/local/apache2//include -c -o mod_maquette.lo mod_maquette.c
touch mod_maquette.slo
mod_maquette.c: In function `maquette_handler':
mod_maquette.c:21: warning: initialization makes pointer from integer
without a cast
/usr/local/apache2//build/libtool --silent --mode=link gcc -o
mod_maquette.la -lapreq2 -rpath /usr/local/apache2//modules -module
-avoid-version mod_maquette.lo
/usr/bin/ld: cannot find -lapreq2
Search libapreq2.so on your disk. Assuming it is located in
/usr/local/lib/libapreq2.so you write
/usr/local/apache2/bin/apxs -c -L /usr/local/lib -l apreq2 mod_maquette.c
Check also that the lib is called libapreq2.so on your system. Maybe
it is libapreq.so (without the '2'). Adapt your command line
accordingly.
S
collect2: ld returned 1 exit status
apxs:Error: Command failed with rc=65536
.
/usr/local/apache2//build/instdso.sh
SH_LIBTOOL='/usr/local/apache2//build/libtool'
mod_maquette.la/usr/local/apache2//modules
/usr/local/apache2//build/libtool --mode=install cp
mod_maquette.la/usr/local/apache2//modules/
cp .libs/mod_maquette.so /usr/local/apache2//modules/mod_maquette.so
cp: cannot stat `.libs/mod_maquette.so': No such file or directory
apxs:Error: Command failed with rc=65536
2010/2/16 Sorin Manolache sor...@gmail.com
2010/2/16 Frédéric Laurendeau frederic.laurend...@gmail.com:
Hi,
I am trying to write a module which will get the request body, process it
and write generated content in the response.
I have seen that some of you used aprec to get the request body but I
just
cannot manage to build my module with it. I'm using apxs to build my
module
and after the build, there is an error when trying to start apache2 with
my
module: Cannot load mod_maquette.so into server: undefined symbol:
apreq_params_as_string
I put at the bottom of the mail my source code and my build script, maybe
I
forgot to include some headers in the source or some libraries in the
build
command line ?
I don't really understand how all of this works but aprec seems very
powerfull and I'd like to find some tutorials or examples on how to build
modules with it.
Thanks for your help.
Frederic
Build script :
/usr/local/apache2/bin/apxs -c mod_maquette.c
Try
/usr/local/apache2/bin/apxs -c -l apreq2 mod_maquette.c
/usr/local/apache2/bin/apxs -i -a -n maquette mod_maquette.la
Source code mod_maquette.c:
#include stdlib.h
#include math.h
#include string.h
#include httpd.h
#include http_connection.h
#include http_config.h
#include http_core.h
#include http_log.h
#include http_protocol.h
#include ap_config.h
#include http_request.h
#include apr.h
#include apr_strings.h
#include apr_lib.h
#include apreq2/apreq_module.h
#include apreq2/apreq_param.h
static int maquette_handler(request_rec *r)
{
const apr_table_t* args;
apreq_handle_t* h = apreq_handle_apache2(r);
apreq_body(h, args);
const char *params = apreq_params_as_string(r-pool, args, NULL,
APREQ_JOIN_QUOTE);
fprintf(stderr, =TEST=\n%s\n==TEST=\n, params );
ap_rprintf(r, Request data read : %s\n, params);
fflush(stderr);
return DONE;
}
static void register_hooks(apr_pool_t *p)
{
ap_hook_handler(maquette_handler, NULL, NULL, APR_HOOK_MIDDLE);
}
module AP_MODULE_DECLARE_DATA maquette_module =
{
STANDARD20_MODULE_STUFF,
NULL, /* dir config creater */
NULL, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
NULL, /* command apr_table_t */
register_hooks, /* register hooks */
};
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
Re: Using mod_rewrite in my authorization module. Need advice.
By the way. The order in which the callbacks are called is translate_name, then authentication/authorisation and last fixups. If you put the RewriteRule at server level then it is applied during the translate_name callback. If you put it inside a Location directive, it is applied during the fixups callback (i.e. _after_ the authentication/authorisation!) S On Wed, Feb 3, 2010 at 22:24, Kevac Marko ma...@kevac.org wrote: Hello. I am porting our authorization module to mpm_worker from mpm_prefork. This authorization module was using some ugly, but nevertheless working hack which messes with mod_rewrite configuration. This authorization module check user permissions for current url A and rewrites url to AA or AB according to permission check result. Because I didn't want to reimplement mod_rewrite and because mod_rewrite don't have any API for url rewriting, this ugly hack was made. I am changing mod_rewrite configuration on the fly, putting A-AA or A-AB RewriteRule to it. I hope you understood that. Ok. This works. But only in mpm_prefork, where only one execution thread exists in apache process. Now I need same functionality, but in mpm_worker. And I don't have idea how to do it easily. Only two ideas came into my mind: 1) Reimplement needed mod_rewrite functionality in out authorization module. 2) Patch or fork mod_rewrite module. Implement some API (like rewrite(from, to);) which could be used for url rewriting with full mod_rewrite power. 3) Patch mod_rewrite and enclose RewriteRules configuration with locks. Sick! Ugly! What do you think? I am looking forward for comments and advices. Thanks. -- Marko Kevac Sent from Moscow, Mow, Russia -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: Bandwith limit/per user from mysql
On Tue, Dec 8, 2009 at 05:11, partysoft partys...@gmail.com wrote: I am looking for a solution to limit the bandwith for the users of a site that have access to some mp3 / subscription. I don't want to serve files through PHP, but directly with some apache module.. do i have to count every bit? or how this should be handled. I have all the info on a mysql DB. Thank you so much for your replies. This kind of functionality, in my opinion, is better implemented at transport level and not at application level. I've done something similar, but more primitive, i.e. the cumulated bandwidth of all connections to port 80 was capped, using traffic shaping in the linux kernel. The network packets are classified with iptables and then each class can be given a different queueing policy with tc. Check http://lartc.org/howto/, chapter 9. I didn't do it per user, but I suppose one can easily do it per client IP address. I admit that maybe this approach is not suitable when you identify users with some data token at application level. S
Re: Segfault doing SQL select
On Tue, Dec 1, 2009 at 09:26, Micah Yoder mi...@yoderdev.com wrote:
Hello,
I'm new to this. Just read most of Nick Kew's book and am trying to write a
module that is a thin-as-possible layer between an AJAX type rich client web
app and an SQL database. I then hope to build a small CMS on top of that.
Currently I'm running into a road block with a segfault every time I run the
query. I'm using Postgres 8.4 from the Ubuntu repos and a custom-compiled
Apache 2.2.14 with the Worker MPM. Here is the relevant code:
void sql_template(ap_dbd_t *con, request_rec *r, apr_hash_t *formdata) {
apr_dbd_results_t *res;
Set res to NULL before passing it.
apr_dbd_row_t *row;
Set row to NULL before passing it.
int rv;
rv = apr_dbd_select(con-driver, r-pool, con-handle, res, select * from
topics;, 0);
if (!rv)
ap_rputs(Failed to run query.\n, r);
while (apr_dbd_get_row(con-driver, r-pool, res, row, 0) != -1) {
ap_rputs(*** Row data *** , r);
ap_rputs(apr_dbd_get_entry(con-driver, row, 0), r);
ap_rputs( ... , r);
ap_rputs(apr_dbd_get_entry(con-driver, row, 1), r);
}
}
This is being called from the main handler function.
apr_hash_t *formdata;
ap_dbd_t *con;
[...]
if (r-method_number == M_GET)
formdata = parse_form_from_string(r, r-args);
[...]
con = ap_dbd_acquire(r);
if (!con) return HTTP_INTERNAL_SERVER_ERROR;
sql_template(con, r, formdata);
Kind of ugly but I'm just trying to get the concepts to work.
I've run it through GDB and sometimes the segfault is on apr_dbd_select line
and sometimes it is on the apr_dbd_get_row line. The arguments passed into
my sql_template function seem to be valid.
Am I doing anything obviously wrong? Or can someone point to a simple
handler module that does a SELECT? (mod_auth_dbd appears to use a different
method and introduces other complexities.)
Thanks,
Micah
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
Re: post-config phase question
On Tue, Nov 10, 2009 at 23:40, Michael Durket dur...@highwire.stanford.edu wrote: Based on the few books that describe Apache module writing and a presentation that I've found on the web from an Apache conference, the advice to module writers is to remember that Apache calls the post-config phase twice - once while it's checking its configuration files, and then when it's ready to start up it discards all that and calls post-config again to really set things up. But it appears to me (via tracing I've done in my own module) that post-config is actually called once at configuration file checking time, and then once per server process start (we're using MPM here) during the second (actual Apache startup phase). Is this correct? No. It's done twice, before forking the children. The post_configs run as the user who launched apache (typically root), before the sever switches to the user/group specified in the corresponding configuration directives. Have a look in server/main.c in the sources of apache. Sorin
Re: Repetitive warnings on ErrorLogs.
On Tue, Sep 8, 2009 at 13:54, Jaysingh Samuel jayasingh.sam...@hotmail.comwrote: Hi soren, Thanks for useful information on this. We are not spawning children from the pre/post config, but still i could see the long lost child alerts. This alerts is coming only on graceful restart(SIGUSR) of the server and not on restart on the server. When i did debug on the worker server_main function i could see that after the run config is done for all the modules, there are many process ID thrown with long lost child, and after that the server resumes with the children. Iam wondering is the alerts are showing because of the custom modules what we are using a longer time to finish with?. Well, I have no idea, but it could be the cause. Try putting a big GracefulShutdownTimeout http://httpd.apache.org/docs/2.2/mod/mpm_common.html#gracefulshutdowntimeoutand see what happens. S Please throw some light on this. Thanks in advance. Jaysingh Samuel Date: Mon, 7 Sep 2009 08:51:25 +0200 Subject: Re: Repetitive warnings on ErrorLogs. From: sor...@gmail.com To: modules-dev@httpd.apache.org On Mon, Sep 7, 2009 at 08:41, Jaysingh Samueljayasingh.sam...@hotmail.com wrote: Hi, We are getting the following warning's on the error Logs. The warning are shown at graceful restart of apache server. [warn] long lost child came home! (pid 31188) We are doing graceful restart of apache server every 30 minutes and we get these warning repetitively, i have tested with the apache restart and that's not showing warning. Is this a fatal error?. More light on this regarding the cause and solution to stop this will be very helpful. Thanks in advance,Jaysingh Samuel. Apache keeps its children in a table of process IDs, called the scoreboard. Then, the main server, the one that has created and started the children processes, waits on them. When the child exits, the main server looks up the PID of the exiting child in the scoreboard. If the child is not in the scoreboard, you get this warning. It may occur if you start children from your main server (for example in the pre/post_config hooks). As you start them yourself, they won't be managed by apache, so their PIDs do not appear in the scoreboard. This would be a harmless way in which you could get the warning when you shut down your children. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail? _ We all see it as it is. But on MSN India, the difference lies in perspective. http://in.msn.com -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: apr_shm_attach fails with permission denied
On Wed, Sep 2, 2009 at 14:58, Robert Schulzer...@bytecamp.net wrote: Hi, a module creates a named shared memory segment in the post_config hook. During child initialisation, apr_shm_attach is called, which in my case fails with permission denied. Is this considered normal behaviour? How can I deal with that? with kind regards, Robert Schulze Does it work if you start apache as root? S
Re: Closing KeepAlive connection - for server push
On Fri, Aug 21, 2009 at 11:28, Alexander
Farberalexander.far...@gmail.com wrote:
Hello,
I have a multiplayer flash game talking to an Apache module
(and the module in turn talks over a Unix pipe to the backend-daemon).
It works ok, but is naturally a bit sluggish because updates
are being polled by the flash client every 20 seconds.
I'd like to rewrite my application, so that a HTTP request
hangs if there is no new information and returns later -
when new infos from other players arrive at the server.
http://code.google.com/p/google-web-toolkit-incubator/wiki/ServerPushFAQ
recommends to close the KeepAlive connection
when returning the new information to the browser.
How can I forcibly close the KeepAlive connection from my module please?
Currently I open connection to the Unix pipe in
the child init phase and then in the handler phase
I read the GET or POST request, write to the pipe,
read back from the pipe and write to the browser:
ap_set_content_length(r, len);
ap_send_http_header(r);
if (len != ap_rwrite(buf, len, r)) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
ap_rwrite (%u bytes) failed, len);
return HTTP_INTERNAL_SERVER_ERROR;
}
return OK;
Thank you for any ideas
Alex
PS: I'm using Apache 1.3.29 on OpenBSD 4.5 with
KeepAlive On and KeepAliveTimeout 300
I have not tried it, but try
r-connection-keepalive = AP_CONN_CLOSE;
S
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
Re: [Fwd: Re: Having some issues with subrequests and filters]
On Tue, Aug 18, 2009 at 08:38, Michael Spieglem...@nauticaltech.com wrote:
Nick Kew wrote:
Michael Spiegle wrote:
Hello,
I have a module I am writing which implements a content-generator and
a filter. If I make a sub-request from within my content-generator,
the filter's (request_rec*)f-r only points to the original request.
How do I access the current sub-request from within my filter?
How do you know the subrequest exists at the point where you access
it from the filter?
Put whatever you need on your module's filter's context.
I was hoping that when my filter starts receiving data for a subrequest, the
request_rec* attached to the filter would indicate that it was a subrequest
via f-r-main not being NULL. It seems like you can use that logic
everywhere else, so why not in the filter context?
I use the following code:
request_rec *newreq = ap_sub_req_lookup_uri(url, req, NULL);
ap_filter_t *flt = ap_add_output_filter(FILTER_NAME, NULL, newreq,
newreq-connection);
// flt-ctx = flt_ctx;
int ret_code = ap_run_sub_req(newreq);
ap_destroy_sub_req(newreq);
The filter is
apr_status_t output_filter(ap_filter_t *f, apr_bucket_brigade *bb) {
// f-r-main is not NULL !
}
This code works for me.
I hope this helps.
S
--
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
Re: mod_deflate feature needed
On Thu, Jul 16, 2009 at 00:39, Anthony J. Biaccoabia...@formatdynamics.com wrote: I'm trying to use mod_deflate to compress data coming out of tomcat through mod_jk and need the proper content-length header set for the COMPRESSED data, but can't do this because the data is streamed and sent after the headers are set, therefore we don't know the compressed content-length until after the fact. I'd either like to request a option to enable such a feature where I can have the compressed data buffered, the headers set, and then the data sent. I'd be willing to pay someone here a nominal free to do it privately, if one so wishes to do it. Have you tried placing the JkMount directive inside a Location and using just one argument for JkMount? LocationMatch /examples/* JkMount ajp13_worker SetOutputFilter DEFLATE /LocationMatch In my case it works, I get a compressed response. Here are the relevant response headers: HTTP/1.1 200 OK Date: Thu, 16 Jul 2009 16:28:19 GMT Server: Apache/2.2.11 (Debian) mod_jk/1.2.26 mod_perl/2.0.4 Perl/v5.10.0 Accept-Ranges: bytes ETag: W/1127-1245356448000 Last-Modified: Thu, 18 Jun 2009 20:20:48 GMT Vary: Accept-Encoding Content-Encoding: gzip Transfer-Encoding: chunked Content-Type: text/html As the transfer is chunked, there's no Content-Length but the response is still compressed. Are you constrained not to use chunked transfer? Sorin -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: Memory leaks on adding module directives on virtual host.
On Wed, Jul 8, 2009 at 08:48, Jaysingh Samueljayasingh.sam...@hotmail.com wrote: Hi all, Please help me out in this. I have Name based virtual host and i place lots of directives from which my modules takes input. Because of adding these directives i face memory leak.. The httpd memory size gets increased on every graceful restart.. My virtual host looks like this. VirtualHost *:80 ServerName click.xyz.com ServerAlias t1.xyz.com xyzApache abc apachedefCookieName def apachezxyCookieName zxy /VirtualHost Please let me know why the graceful restarts is not cleaning up memory on the virtual hosts and ways to prevent memory leaks. Thanks, Jaysingh Samuel Each graceful restart apache runs the pre- and post-config callbacks as well as the callbacks associated to the configuration hooks. Check that you do not have malloc's and new's in those callbacks. The same applies to other resources like shared memory segments, semaphores, application specific threads, etc. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
