Re: apche rewrite and reverse proxy help
On Sun, Oct 14, 2012 at 1:12 AM, Abdel kad...@yahoo.com wrote: I have back end website running on Tomcat with the following url http://local.domain.com/app. External user access the website through apache proxy with the following url http://www.domain.com/user1 (user1, user2, etc... It’s uri specific to each user). I want to use apache rewrite or/ and reverse proxy directive to translate the url like http://www.domain.com/user1 into http://local.domain.com/app?user=user1 Please can someone help me please? You're on the wrong mailing list. Try users.
Re: RPC over HTTP
On Wed, Oct 3, 2012 at 11:57 AM, Evgeny Shvidky evg...@skyfence.com wrote: Hi, I am developing a new module on C. One of the requirements of my module is to receive and handle RPC over HTTP protocol. RPC over HTTP opens two HTTP/1.1 requests: One with request method RPC_IN_DATA to send data to the server, and second one with method RPC_OUT_DATA to send data back to the client. The body consists of raw binary data, and the connections are apparently re-used for several RPCs. Here's an example of an IN connection header: RPC_IN_DATA /rpc/rpcproxy.dll?exchange:6002 HTTP/1.1 Content-Length: 1073741824 ... After connections are established client sends on IN channel RPC message with 104 bytes. I use the following apache API's in order to read these client message. ap_setup_client_block(userReq, REQUEST_CHUNKED_ERROR); ap_should_client_block(userReq) ap_get_client_block(userReq, buf, size) The problem is ap_should_client_block function returns 1 (means there is message to read) but ap_get_client_block returns error (-1) and nothing read. Do you check that ap_setup_client_block() returns OK? Does ap_get_client_block() return -1 on the first call? If yes, you may want to step through the function in a debugger to see what the error condition is. How should I read client's data? Is there any other API for it? The filter API?
Re: RPC over HTTP
On Wed, Oct 3, 2012 at 2:06 PM, Evgeny Shvidky evg...@skyfence.com wrote: Hi, ap_setup_client_block() returns OK. I think the problem is that Content-Length header value is 1073741824 (1 GB) and probably apache tries to receive the whole content before it passed to my module. Am I right? If yes, is there any way to tell apache to send all received data till now? ap_get_client_block() tries to read up to the number of bytes you requested. Now, if you passed in the value of the Content-Length header, then it will either stall for a long time or simply fail.
Re: aprmemcache question
On Thu, Sep 27, 2012 at 4:05 AM, Joshua Marantz jmara...@google.com wrote: RE failing the build of my module -- the dominant usage is via precompiled binaries we supply. Is there an apr query for determining whether apr was compiled with threads I could do on startup? I don't think there's an official way but you know apr was compiled with APR_HAS_THREADS when dlsym(NULL, apr_os_thread_current) != NULL. Using dlsym() like that is not quite compatible with POSIX but it works on all the major Unices.
Re: aprmemcache question
On Thu, Sep 27, 2012 at 4:29 PM, Joshua Marantz jmara...@google.com wrote: Thanks Ben, That might be an interesting hack to try, although I wonder whether some of our friends running mod_pagespeed on FreeBSD might run into trouble with it. I did confirm that my prefork build has APR built with APR_HAS_THREADS, which for some reason I had earlier thought was not the case. It should work, provided you linked against libapr. The FreeBSD man page says this: If dlsym() is called with the special handle NULL, it is interpreted as a reference to the executable or shared object from which the call is being made. Thus a shared object can reference its own symbols. And that's how it works on Linux, Solaris, NetBSD and probably OpenBSD as well. Do you have a feel for the exact meaning of that TTL parameter to apr_memcache_server_create? You mean what units it uses? Microseconds (at least, in 2.4).
Re: Input filters that alter content-length
On Tue, Sep 25, 2012 at 1:15 PM, Ivan Prostran ivan.prost...@bulb.hr wrote: Hi, I have the following scenario : - Apache/2.2.19 (Solaris 10 SPARC) SetInputFilter alterxmlbody (AP_FTYPE_RESOURCE) SetHandler weblogic-handler The handler forwards requests to multiple weblogic servers and the filter itself analyzes and/or changes the POST data in a way that the length of the incoming request may be changed. The question is : What is the proper and legitimate way to deal with this situation? Should the input filter set the new CL value in headers_in before returning from the callback, or just remove it from the list? apr_off_t length; apr_brigade_length (bb,1,length); apr_table_setn(f-r-headers_in, Content-Length, apr_off_t_toa(f-r-pool, length)); Unfortunately, this approach breaks the pipeline because sometimes the filter needs to buffer data over more than one call, but I do not consider this as a problem. or apr_table_unset(f-r-headers_in, Content-Length); You didn't tell how your handler works (how it forwards the request) but removing the Content-Length header is probably easiest. Does the upstream server understand HTTP/1.1 or HTTP/1.0 with a TE header?
Re: Bucket brigade filter thread safety
On Mon, Sep 10, 2012 at 3:53 PM, Alex Bligh a...@alex.org.uk wrote: Ben, No, but the documentation omits some crucial details. apr_pool_create() is thread-safe only if: 1. libapr is compiled with APR_HAS_THREADS 2. APR_POOL_DEBUG is turned off 3. the parent pool has a thread-safe allocator (which is true for the global allocator that is used when parent=NULL, provided conditions #1 and #2 are satisfied) The pools you get from httpd core satisfy #3 but a module may replace e.g. r-pool with another pool that doesn't. Ergo, don't rely on a pool being thread safe unless you explicitly make it so. Ah, OK. I had thought that was only an issue as when the pool create was running (at which point I am single threaded). But I can fix that - thanks. That won't solve all your problems though. Bucket brigades are not thread safe, you will need something to synchronize on. So what I was trying to do was to use a) the input bucket brigade in thread #1 (main thread) b) the output bucket brigade in thread #2 in an attempt to avoid synchronization But what I don't understand is whether thread #2, in writing to the output filters (which presumably have a reference to r-pool) will need synchronisation. Yes. It's not just because r-pool may or may not be synchronized, the internal structure of the bucket brigade is not protected by any locks either. Oh I understand that, but I thought in the example above only thread #1 would be accessing the input bucket brigade and only thread #2 would be accessing the output bucket brigade, so there would be no need for synchronisation as they were thread local. And if I have to synchronize, how do I do that in practice? Thread #2 does and ap_fwrite/ap_flush so I can hold a mutex there. But what do I do in thread #1, which calls ap_brigade_get and blocks? I can't hold a mutex during that. I can make it a non-blocking ap_brigade_get (if I understood how to do it) Non-blocking reads are pretty straightforward: apr_thread_mutex_lock(mutex); rv = ap_get_brigade(f-next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ, len); if (APR_STATUS_IS_EAGAIN(rv)) apr_thread_cond_wait(cond, mutex); rv = ap_get_brigade(f-next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ, len); apr_thread_mutex_unlock(mutex); The other thread wakes up this thread with apr_thread_cond_signal(cond). I think I may not have explained what I am doing clearly. One thread is doing input (from the apache client), and the other output (to the apache client). The ap_brigade_get is in the input thread (the main thread) and is blocking on the client sending more data. So the thing that would need to wake the thread up is more data becoming ready from the client - nothing to do with the other thread. I don't know how to detect that. but what I really need is the equivalent of a select() which I can do with the mutex not held (or some way to drop the mutex during the raw reads). Any ideas? You could set up a pollset in the main thread and funnel incoming data into your bucket brigade. Not terribly efficient (lots of context switches) but the real world impact may very well be negligible and you can support multi-process setups with zero changes to your code. Hmm, well if I could use a pollset in my main thread I wouldn't need bucket brigades at all. But as this is data coming from the client, surely it's going to be in a bucket brigade already as it will have passed through all the input filters etc., having been read by apache itself? Diagramatically: | APACHE | ... MODULE ... | Client == Apache ap_get_brigade do_something [thread1] ^ |-- ap_fwrite --- do_something_else [thread2] Each thread has a different bucket brigade with a different allocator. ap_get_brigade either needs to block, or if it's non-blocking it needs to wait on a condwait or something for /apache/ to produce more data from the client, not on the other thread. Right, I think I see what you mean. Apache may not be a perfect fit. I once had to solve a similar issue and I eventually settled on sending the socket to another process. Managing mostly dormant connections turned out to be too much of a headache to do from within httpd.
Re: Bucket brigade filter thread safety
On Sun, Sep 9, 2012 at 2:31 PM, Alex Bligh a...@alex.org.uk wrote:
I am trying to work out how to develop a thread-safe module with two
threads, one thread reading and one thread writing. I'm using mpm-prefork
on apache 2.2.14-5ubuntu8.9 in case that matters. The module is a websocket
proxy.
My main thread (#1) is doing (in essence - error handling removed)
apr_pool_create(opool, r-pool);
apr_bucket_alloc_t *oallocator = apr_bucket_alloc_create(opool);
apr_bucket_brigade *obb = apr_brigade_create(opool, oallocator);
apr_thread_create (... otherthread ...);
while (1) {
bb = apr_brigade_create(r-pool, r-connection-bucket_alloc);
ap_get_brigade(r-input_filters, bb, AP_MODE_READBYTES,
APR_BLOCK_READ, bufsiz));
apr_brigade_flatten(bb, buffer, bufsiz);
apr_brigade_destroy(bb);
do_something_with(buffer);
}
The other thread (#2) is doing (in essence)
while (1) {
apr_pollset_poll(recvpollset, timeout, ... );
apr_socket_recv(socket, buf, ...)
do_something_else_with(buf);
ap_fwrite(r-connection-output_filters, obb, ...);
ap_fflush(r-connection-output_filters, obb, ...);
}
I am suffering from very occasional corruption of the bucket brigade which
normally shows up as a corrupted pool pointer or a bogus bucket entry in
thread #1 (for instance a SEGV in apr_brigade_length). Interestingly this
is the relatively quiet input brigade which is only ever touched by the
main apache thread. It's almost as if an allocator is not thread safe.
That's because it isn't unless you explicitly make it so (which no MPM does).
However, I'm using a separate bucket allocator (see code above) and (at
least in my code) a separate pool.
They're not really separate, the sub pool is created off r-pool. You
should probably use apr_pool_create_ex() here with parent=NULL and an
allocator that you created with apr_allocator_create() +
apr_allocator_mutex_set().
That won't solve all your problems though. Bucket brigades are not
thread safe, you will need something to synchronize on.
Could the output filter chain somehow be using the allocator attached to
the request (i.e. thread #1)? If so, how can I stop this? I can't run the
ap_fwrite/ap_flush under a mutex and have the same mutex held during
ap_get_brigade, as the latter blocks (and I can't see how to use the
non-blocking version without spinning).
[apologies for the partial dupe on the apache-users mailing list - I think
I've got nearer the problem since then and this list seems more
appropriate]
--
Alex Bligh
Re: problem with ap_md5 in custom module
On Wed, Aug 15, 2012 at 5:13 PM, nik600 nik...@gmail.com wrote:
Dear all
i'm having a problem with ap_md5, i just want to write a custom module
that compute che md5 checksum of the requested url and give it back to
the user.
This is my code:
*
*
static int kcache_handler(request_rec* r)
{
if (!r-handler || strcmp(r-handler, kcache))
return DECLINED;
if (r-method_number != M_GET)
return HTTP_METHOD_NOT_ALLOWED;
char* kcache_md5;
kcache_md5 = (char *)ap_md5(r-pool,r-unparsed_uri);
ap_set_content_type(r, text/html;charset=ascii);
ap_rputs(!DOCTYPE HTML PUBLIC \-//W3C//DTD HTML 4.01//EN\, r);
ap_rputs(htmlheadtitleK-Hello World!/title/head, r);
ap_rprintf(r,bodyh1K-Hello
World!/h1p%s=%s/p/body/html, r-unparsed_uri,kcache_md5);
return OK;
}
*
*
i've got a warning during compilation:
src/mod_kcache.c:18:15: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]
Is quite strange to me that ap_md5 returns an int, as in the
documentation it is reported to return a char *
http://ci.apache.org/projects/httpd/trunk/doxygen/group__APACHE__CORE__MD5.html
By the way, if i try to run it i get a segfault, if i comment the line
that prints kcache_md5 with ap_rprintf the module doesn't segfault.
So, where i'm wrong?
It seems you don't include util_md5.h so the compiler defaults the
function prototype to `int ap_md5()`. Compile with -Wall -Wextra and
you'll get warnings about things like that.
Re: Anyone have some example code doing simple HTTP GET request from within a module?
On Sat, Jun 23, 2012 at 4:47 AM, oh...@cox.net wrote: Per earlier threads on this list, I've been working on an Apache module. For the time being, I'm kind of stuck because of the problems that I've run into with trying to integrate my module with a 3rd party library, so just for my module, which is mainly a proof-of-concept, I'd like to have my module do an HTTP GET request. So, I was wondering if anyone has some simple example code for doing that from within a module, maybe using libcurl, or just natively using sockets? I'm trying to do this myself, and I've been looking at using libcurl, but most of the examples that I've seen use the easy setup, so if someone has something like that that can be shared, it'd be a big help. Conversely, if I figure it out, I'll post some working snippets here :)... I suggest you reuse the existing infrastructure where possible. Have a look at how mod_proxy makes HTTP requests.
Re: How to compiling/link/use Apache module that uses shared library?
On Thu, Jun 21, 2012 at 8:43 PM, oh...@cox.net wrote: I tried that, which allowed me to start Apache, but am getting a segfault. Run it through gdb and inspect the backtrace. Compiling with debug symbols and optimizations disabled (-g -O0) will help.
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache module that uses shared library?
On Fri, Jun 22, 2012 at 3:32 AM, oh...@cox.net wrote: Program received signal SIGSEGV, Segmentation fault. 0x003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4 So, it's actually blowing up in BN_num_bits() in /lib64/libcrypto.so.4? Type `bt full` and you'll get a backtrace + locals. That should tell you where the error originates from and maybe why it happens.
Re: Best (safest) way to edit char string (from envvars)?
On Wed, Jun 20, 2012 at 4:35 PM, oh...@cox.net wrote: Hi, I am working on a module, and I get one of the SSL envvars, SSL_CLIENT_CERT, using apr_table_get() into a const char *. The client cert char string returned has the extra beginning line (-BEGIN CERTIFICATE-) and ending line (-END CERTIFICATE-), but I need to remove both of those lines for a call that I need to make. I have to admit, I'm a bit (a lot) rusty with 'C', and I guess I could do something like: strpy(original_cert, original_cert+27); and then set the ending position to \0 (to terminate the char string early), but since with this is a module, and I'm working with a pointer to the memory pool, I'm kind of worried that doing stuff like that would mess things up (e.g., garbage collection, since the string is now shorter by 'x' bytes. So, from an Apache module development standpoint, what would be the safest way to do this (strip a string of chars from the beginning and end)? Make a copy with apr_strdup(), then mutate the copy. APR has utility functions for manipulating strings, like apr_strtok(). Have a look at apr_strings.h.
Re: How to access client certificate PEM and incoming request headers in a module?
On Mon, Jun 18, 2012 at 8:53 AM, oh...@cox.net wrote: I added a call to header_request_env_var(r, REMOTE_URI), just to see what it got (running Apache in single-process mode): printf(REMOTE_URI=[%s]\n, header_request_env_var(r, REMOTE_URI) ); Then I pointed a browser to http://myhost/test, where /test was a Location with a RequestHeader (to trigger mod_headers) but I got: REMOTE_URI=[(null)] Shouldn't that be showing: REMOTE_URI=[/test] ?? Did you mean REMOTE_USER or REQUEST_URI? I don't think there's such a thing as REMOTE_URI.
Re: How to access client certificate PEM and incoming request headers in a module?
On Sun, Jun 17, 2012 at 9:46 PM, oh...@cox.net wrote: Hi, I am starting to look into implementing an Apache module that can use information from an incoming request, including several headers and the subject string from a client certificate to do authentication. I've been looking at the source for mod_auth_certificate, from https://modules.apache.org/, as a starting point. However, it looks like the way that mod_auth_certificate works is that it requires that there's an SSLUserName directive to put the client certificate DN into the Apache REMOTE_USER attribute, whereas I need the entire PEM for the client cert to do authentication that I'm trying to do. So I was wondering if it's possible for a module to access the SSL_CLIENT_S_DN and SSL_CLIENT_CERT environment variables, and if so, how? They should be set in r-subprocess_env provided `SSLOptions +StdEnvVars +ExportCertData` is set in the server or per-directory config. Also, as mentioned my module would need to access several HTTP headers that are in the incoming requests. How can it do that? Look them up with `apr_table_get(r-headers_in, X-Header-Name)`.
Re: How to access client certificate PEM and incoming request headers in a module?
On Mon, Jun 18, 2012 at 5:45 AM, oh...@cox.net wrote:
I haven't actually tried your suggestion yet, but, re. the SSL variables, I
was looking at mod_headers.c, and in there, there are two separate functions:
static const char *header_request_env_var(request_rec *r, char *a)
{
const char *s = apr_table_get(r-subprocess_env,a);
if (s)
return unwrap_header(r-pool, s);
else
return (null);
}
static const char *header_request_ssl_var(request_rec *r, char *name)
{
if (header_ssl_lookup) {
const char *val = header_ssl_lookup(r-pool, r-server,
r-connection, r, name);
if (val val[0])
return unwrap_header(r-pool, val);
else
return (null);
}
else {
return (null);
}
}
So, it seems like the method to get the SSL variables is different than the
other environment variables?
Or, does setting SSLOptions the way that you suggested cause the SSL variable
so also exist in apr_table_get(r-subprocess_env, )?
Oh, I forgot about that. It's the ssl_var_lookup optional function,
that might even work without having to tweak SSLOptions.
Re: Get the directory of the module
On Mon, Jun 11, 2012 at 10:10 PM, Bart Wiegmans bartwiegm...@gmail.com wrote: Hello everybody, For a project I'm doing, I need to install a few bytecode files alongside my module. I was planning on placing them in the modules directory but I realised that at runtime I do not know where that is. What is more, that directory may (and will, as a matter of fact) vary during installation and testing. But most importantly, the server knows where the module is kept as it specified by LoadModule. So in short, how can I determine the directory of my module at runtime? You can't, the file path is internal to mod_so.c. I don't think it even stores it.
Re: Change Request-Header before mod_rewrite
On Mon, Jun 4, 2012 at 10:53 PM, Marc apocalypse17 apocalyps...@gmx.de wrote:
Hi all,
I just developed my first apache module following the tutorial on the apache
website. The module is responsible for adding one header value to the active
request which must be checked in a mod_rewrite ReWriteCondition.
The problem is, that this value never reaches the mod_rewrite Rule. The
Header just behaves the same as the original request. Does anyone know why?
What am I doing wrong?
My module looks like this:
static int helloworld_handler(request_rec* r){
if (!r-main) {
apr_table_setn(r-headers_in, X-CUSTOM-HEADER, 1);
}
return DECLINED;
}
static void register_hooks(apr_pool_t* pool){
ap_hook_handler(helloworld_handler, NULL, NULL, APR_HOOK_FIRST);
}
module AP_MODULE_DECLARE_DATA helloworld_module = {
STANDARD20_MODULE_STUFF,
NULL,
NULL,
NULL,
NULL,
example_directives,
register_hooks
};
The .htacces file looks like this:
RewriteEngine on
RewriteCond %{HTTP:X-CUSTOM-HEADER} 1 [NC]
RewriteRule from.html to.html
The Rewrite-Rule is never executes fine. It always show the content of
from.html.
Thank you in advance,
Marc
mod_rewrite.c does most of the interesting work in a translate_name
hook. By the time your handler hooks run, it's probably too late.
Re: mod_ssl ignores connection-aborted eos_sent
On Tue, Mar 6, 2012 at 13:27, Daniil A Megrabjan
danil.megrab...@gmail.com wrote:
Anyway, as far as I understood I'm not allowed to change the default
behavior of mod_ssl. In this case there is the other question - how to
register my hook to be really before mod_ssl? and even if request has
been received on 443 TCP port process the request as usual HTTP.
Something like:
*
*
*static* *const* *char* * *const* aszPre[] = { mod_ssl.c, NULL };
ap_hook_handler(ixcell_init_handler, aszPre, NULL, APR_HOOK_REALLY_FIRST);
doesn't help.
Try ap_hook_pre_connection().
Re: mod_ssl ignores connection-aborted eos_sent
On Mon, Mar 5, 2012 at 22:34, Daniil A Megrabjan danil.megrab...@gmail.com wrote: Hello, I'm writing a module which serves a special URL. In cases when URL-string matches the special pattern my module sends the connection(SCM_RIGHTS) between HTTP client and Apache to another process. Furthermore, Apache child has been told to forget about this connection in this way: r-connection-aborted = 1; r-eos_sent = 1; After that my process communicates with HTTP-client by itself without Apache's assistance. Everything is fine with this scheme inside basic HTTP, but when I'm switching to HTTPS I can guess that mod_ssl ignores aborted and eos_sent properties and eventually drops the connection. How to persuade mod_ssl not to touch the connection? I don't think you can - or should. How will you decrypt the traffic? The SSL/TLS session parameters are private to mod_ssl.
Re: thread ID
On Thu, Mar 1, 2012 at 17:29, sorin.manola...@orange.com wrote: Hello, I would need a memory buffer associated per worker thread (in the worker MPM) or to each process (in the prefork MPM). In order to do that, I would need a map thread-buffer. So, I would need a sort of thread ID/key/handle that stays the same during the lifetime of the thread and no two threads in the same process can have the same ID/key/handle. What is the most portable way to get this thread ID? I thought of r-connection-id. It works but it is not very portable as it is not guaranteed that two connections created by the same thread will have the same id. They do for now. If r-connection-sbh was not opaque it would be great, because sbh-thread_num would be exactly what I need. I could also use pthread_self. It works too but, in general, it is not guaranteed that the worker threads are pthreads. Thank you for your help. Sorin What about apr_os_thread_current()? It returns a opaque value that's a pthread_t on Unices and a pseudo-HANDLE on Windows. Read this[1] to understand what that means. As a recovering standards lawyer I should probably point out that pthread_t is an opaque type that's not guaranteed to be convertible to a numeric value (or to anything, really). That said, I've never seen a pthreads implementation where that wasn't the case. [1] http://msdn.microsoft.com/en-us/library/windows/desktop/ms683182%28v=vs.85%29.aspx
Re: Performance Evaluation of a Module
On Wed, Feb 8, 2012 at 14:01, Oğuzhan TOPGÜL oguzhantop...@gmail.com wrote: Hi all, I have developed an apache module and i want to evaluate the performance of my module. I want to see how my module increases the load. I want to measure the effect of my module on processor and memory. I decided to set an evaluation environment using snmp and cacti. I'm sending thousands of request from a laptop to the server which my module is installed by using apache benchmark (ab). And i'm measuring load average, memory usage data from cacti in two different cases (my module is active and passive) Do you think this is a good evaluation environment or do you have any ideas, suggestions??? sincerely Oguzhan That's a pretty sensible approach. Measuring CPU usage is usually better done with a profiler unless a coarse-grained indicator like the load average is good enough.
Re: Developing Authn/Authz Modules
On Sat, Oct 1, 2011 at 23:05, Suneet Shah suneetshah2...@gmail.com wrote:
Hello,
I am trying to build my apache module which needs to carry out
authentication and authorization functions based on the value of a cookie.
To start with, I have just created a shell with the intent that I wanted the
functions for authentication and authorization being called.
However, it does not appear that these functions are being called. I have
pasted by configuration and code below.
When I try to access http://localhost/test_rpc/ I get the login.html that
is defined in my ErrorDocument below.
But when I look in the log file, I see the following.
Since its looking for a userId, I am wondering if there is an error in my
configuration
[Sat Oct 01 16:37:29 2011] [debug] prefork.c(996): AcceptMutex: sysvsem
(default: sysvsem)
[Sat Oct 01 16:38:08 2011] [error] [client 127.0.0.1] access to
/test_rpc/header.jsp failed, reason: verification of user id 'null' not
configured
Any guidance on what I am doing wrong would be greatly appreciate.
Regards
Suneet
-- Configuration in Httpd.conf
Location /
IAM_CookieName IAM_PARAM
IAM_TokenParam tkn
IAM_Service_base_url http://localhost:8080/;
ErrorDocument 401 /login.html
AuthType IAMToken
AuthName IAM Login
AuthCookie_Authoritative On
/Location
Location /test_rpc/
ProxyPass http://localhost:9080/test_rpc
require tkn
/Location
- Module Code
static int authz_dbd_check(request_rec *r) {
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server, authz_dbd_check
called);
return HTTP_OK;
}
static int check_token(request_rec *r) {
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r-server, chedk_token
called.);
return OK;
}
static void authz_dbd_hooks(apr_pool_t *p)
{
ap_hook_auth_checker(check_token, NULL, NULL, APR_HOOK_MIDDLE);
ap_hook_auth_checker(authz_dbd_check, NULL, NULL, APR_HOOK_MIDDLE);
}
module AP_MODULE_DECLARE_DATA authz_dbd_module =
{
STANDARD20_MODULE_STUFF,
authz_dbd_cr_cfg,
NULL,
NULL,
NULL,
authz_dbd_cmds,
authz_dbd_hooks
};
You probably need a `Satisfy all` in your httpd config.
Re: cross-process and cross-thread file locking
On Thu, Sep 29, 2011 at 16:54, thomas bonfort thomas.bonf...@gmail.com wrote: Hi all, sorry in advance if this is a dumb question. The apr documentation for apr_file_lock states Locks are established on a per-thread/process basis; a second lock by the same thread will not block. but this is not the behavior I am seeing. As apr_file_lock on unix uses fcntl by default, a second lock by another thread of the same process will not lock either. You're probably running into (POSIX mandated!) behaviour that requires that when a process closes a file descriptor for file X, *all* locks for X held by that process are released. Absolutely brain dead. I can't begin to fathom the mind that thought it up. I was using apr_file_lock as I need all my httpd threads/process to be synchronized on an named ressource, and chose to create a lockfile who's filename matches my named ressource. This does not work as with a multi-threaded mpm the threads of the same process that created the lockfile will not block on the call to apr_file_lock call. From my readings, it seems that file locking is a hazardous task to get right, so what are my options to attain my goal: - use my own implementation mimicking apr_file_lock, but that unconditionnaly uses flock() instead of fcntl() ? I suspect that this would not be a safe solution as some platforms fall back to fcntl for flock. flock() is not available on SunOS and it has nasty fork() semantics: process acquires lock, forks, child releases lock, parent loses lock (without getting told). Once again, brain dead. You also cannot rely on it working correctly (or at all) on NFS mounts. That's not really flock()'s fault, it's a shortcoming of the NFS protocol. fcntl() and lock() have the same issue. In my experience, the most reliable and portable approach is to create a lock file with open(O_CREAT|O_EXCL) that you unlink() afterwards. On EEXIST, sleep for a bit and try again. - I tried using a posix semaphore which worked quite well, except in the cases where either the process crashed or was terminated by httpd because of a Timeout, and in that case the semaphore is never released until a server reboot or manually messing in /dev/shm. If I attach a cleanup call to the request pool, will it be called in the case where the process is terminated after the Timeout delay ? I don't think you can guarantee that your cleanup action always runs. If a worker process hangs, the master will eventually send it a SIGKILL.
Re: Infinite data stream from a non-HTTPD external process via HTTPD
On Tue, Sep 20, 2011 at 11:13, Henrik Strand henrik.str...@axis.com wrote: I would like to send an infinite data stream from a non-HTTPD external process via HTTPD to the client connection. Both HTTP and HTTPS must be supported. What kind of external process are we talking here? Something that prints to stdout, listens on a UNIX/TCP socket, something else?
Re: po
On Thu, Sep 1, 2011 at 13:52, Joshua Marantz jmara...@google.com wrote: Hi, I've been load-testing our module (mod_pagespeedhttp://code.google.com/speed/page-speed/docs/module.html) with httpd 2.2.16 built with these options: --enable-pool-debug --with-mpm=worker I've been getting periodic aborts from apr_table_addn that don't look like they are from my module. These do not occur when using 'prefork'. Here's a stack-trace recovered from a core file: Program terminated with signal 6, Aborted. #0 0x7fdd3bbd9a75 in raise (sig=value optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) where #0 0x7fdd3bbd9a75 in raise (sig=value optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x7fdd3bbdd5c0 in abort () at abort.c:92 #2 0x7fdd3c9a2e57 in apr_table_addn (t=0xe84980, key=0xd51d60 Accept-Encoding, val=0xd51d71 identity) at tables/apr_tables.c:813 #3 0x00433e36 in ap_get_mime_headers_core (r=0xf27de0, bb=0xdda3a0) at protocol.c:799 #4 0x0043456b in ap_read_request (conn=0xe51620) at protocol.c:918 #5 0x0047f772 in ap_process_http_connection (c=0xe51620) at http_core.c:183 #6 0x00446e28 in ap_run_process_connection (c=0xe51620) at connection.c:43 #7 0x004b3297 in process_socket (thd=value optimized out, dummy=value optimized out) at worker.c:544 #8 worker_thread (thd=value optimized out, dummy=value optimized out) at worker.c:894 #9 0x7fdd3c1339ca in start_thread (arg=value optimized out) at pthread_create.c:300 #10 0x7fdd3bc8c70d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 Questions 1. Is this a bug in httpd? Probably not. 2. Or could I somehow have caused this with a programming error in my module? That seems more likely. 3. Or is enable-pool-debug simply incompatible with the Worker MPM? Not that I know. That assertion is triggered when you add a string from pool A to a table in pool B where A is a child of B (adding headers from the request_rec to a conn_rec table, for example). It's a lifecycle issue.
Re: mutex permission denied
On Wed, Aug 17, 2011 at 15:20, Jason Funk jasonlf...@gmail.com wrote: I am trying to implement an apr proc mutex in my module. When I created the mutex with APR_LOCK_DEFAULT the mutex is successfully created but I am getting Permission Denied when I try to acquire the lock. I ran apr_proc_mutex_defname to get the name of the default mutex type and it is APR_LOCK_SYSVSEM. Without changing anything else, I changed APR_LOCK_DEFAULT to APR_LOCK_POSIXSEM when creating the mutex, just to see what happens and everything works fine. Of course, this isn't portable. Any ideas why APR_LOCK_SYSVSEM doesn't work, but APR_LOCK_POSIXSEM does work? I suspect that you see shmget() raising EACCES when you strace apache? That's what happens when you create the mutex as root and try to acquire it after httpd's dropped privileges, apr creates the semaphore with mode 0600.
Re: mod_proxy and modified headers in filters.
On Mon, Aug 8, 2011 at 10:29, Zaid Amireh tum...@gmail.com wrote: I'm writing a module for Apache 2.2 that changes the content and thus needs to set a new C-L header, all is working perfectly for static files and content generated from content handlers (PHP Ruby Passenger Phusion), an issue arose when testing with mod_proxy, it seems that any changes the module does to the HTTP headers are being ignored by mod_proxy. mod_proxy keeps serving the headers it first got from the backend source and disregards any changes my module does, is it possible to change the headers in this case? Yes. Have a look at proxy_hook_fixups() in mod_proxy.h.
Re: Sharing information between threads and processes.
On Thu, Jul 21, 2011 at 13:25, Zaid Amireh tum...@gmail.com wrote: On Jul 21, 2011, at 1:53 PM, Nick Kew wrote: How near do the socache modules come to meeting your needs? mod_disk_cache would unfortunately make my code pretty complex and maybe slow as I'm not caching documents but rather tokens and strings. mod_mem_cache is a per-process cache which simply doesn't meet the requirements. Not the same thing. The socache API is a 2.3.0 addition that allows you to store blobs in shared memory, memcache, distcache or DBM.
Re: proxying another protocol to http/https
On Thu, Jul 7, 2011 at 02:56, Jodi Bosa jodib...@gmail.com wrote:
I would like to leverage mod_proxy and mod_proxy_http to proxy client
requests (from another protocol).
Assuming I have input output filters that handle the other protocol with
the client, shouldn't I simply be able to:
Handler
{
r-filename = apr_psprintf(r-pool, proxy:https://%s;, hostname);
r-proxyreq = PROXYREQ_PROXY;
return DECLINED; /* to allow mod_proxy to kick in and do it's thing */
}
That should work if your handler runs before mod_proxy. Hook it at
APR_HOOK_REALLY_FIRST.
Re: illegal instruction 4
On Thu, Jul 7, 2011 at 16:33, MK m...@cognitivedissonance.ca wrote:
I have a mod_perl based module running a service on an openVZ slice.
It was working fine for a few weeks, but when I went to use it today I
get delivered an empty page and in the apache error.log:
child exit signal Illegal instruction (4)
Which AFAIK is a very strange thing (SIGILL); actual perl errors are
usually explicit, and passed on from the interpreter. To make sure the
problem wasn't in my code, I replaced the module with a one liner:
sub handler {
return SERVER_ERROR;
}
Same thing. Ie, suddenly perl modules are working. I did not compile
apache or mod_perl myself.
Anyone have any ideas about how I can solve this or debug it further?
I have been playing around with small max stack sizes (ulimit -s 256),
but resetting that to 8192 and restarting apache did not alleviate the
problem.
I'm not sure if this is the right mailing list for you but if you want
to debug Apache, start it in single-process mode (`httpd -X`) and
attach `gdb` to it. That may be only nominally useful if your Apache
is compiled without debug symbols (unless you get a kick out of
stepping through assembly code).
Re: creating reverse proxy workers dynamically
On Thu, Jul 7, 2011 at 07:19, Jodi Bosa jodib...@gmail.com wrote: It seems I may need to create HTTPS reverse proxy workers DYNAMICALLY - what is best way to do this? In other words, from manual I see config directive: ProxyPass /example http://backend.example.com connectiontimeout=5 timeout=30 However, I will have several origin servers that aren't necessarily known during config or startup. How can I create such workers as needed? I don't think there is a reliable way to do that right now.
Re: Module External Configuration
On Tue, Jun 21, 2011 at 23:26, Jason Funk jasonlf...@gmail.com wrote: One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? As I've said before, no, you cannot portably resize a shared memory segment. APR doesn't even expose that functionality. If you're targeting Unices only, you can use Sys V or POSIX IPC: you open() or shm_open() a memory segment, then ftruncate() it to the desired size. Make sure to wrap the call to ftruncate() in an exclusive lock or bad things will happen.
Re: Socket transfer from Apache httpd to a non-httpd process
On Thu, Jun 16, 2011 at 10:32, Henrik Strand henrik.str...@axis.com wrote: I've tried writing data to the socket directly after my non-httpd daemon process receives the socket descriptor and this results in that the client receives this data. However, very shortly afterwards the connections is closed and I'm not able to write to the socket anymore. You probably want to dup() the socket fd before passing it to the external process.
Re: Vary:User-Agent, best practices, and making the web faster.
On Sun, Jun 5, 2011 at 21:37, Joshua Marantz jmara...@google.com wrote: Does Magento actually vary the content of CSS JS based on user-agent? Or does it only vary the content of HTML? I don't know. I'm by no means a Magento expert, I only run into it from time to time. That site I broke? That was in the summer of 2009 while beefing up the security and performance of a large retailer's web shop, mostly by putting stuff behind reverse proxies.
Re: Vary:User-Agent, best practices, and making the web faster.
On Sun, Jun 5, 2011 at 13:42, Joshua Marantz jmara...@google.com wrote: This is a case where the content varies based on user-agent. The recommendation on the mod_deflate doc page is add vary:user-agent for any non-image. Can you think of a case where the absence of a vary:user-agent header causes broken behavior when the content doesn't vary? I'm not objecting to setting vary:user-agent when content varies: that's what it's for. I'm objecting to setting vary:user-agent when content does *not* vary. The mod_deflate documentation unambiguously recommends setting vary:user-agent, and my feeling is that this is to work around a bug that exists only in IE5 or pre-2007 patch of IE6. Sorry, Joshua, we're conflating things. You raised two issues in your original post: 1. Updating the mod_deflate documentation. Seems reasonable. The Vary: UA recommendation was added in 2002 during a general clean-up of the mod_deflate documentation and the commit log doesn't tell why. You could open a bugzilla issue or raise it on the httpd-dev mailing list (the former is the proper channel but the bugzilla is something of a graveyard). 2. mod_pagespeed second-guessing the user's intent. That still seems like an unambiguously bad idea. To touch on Magento again, its documentation links (or linked) directly to that section of the mod_deflate docs and people are using that. If your module scans for and neutralizes that Header directive, you will break someone's site.
Re: Vary:User-Agent, best practices, and making the web faster.
On Sat, Jun 4, 2011 at 21:26, Joshua Marantz jmara...@google.com wrote: I think what we'd do is basically let mod_pagespeed ignore Vary:User-Agent if we saw that it was inserted per this exact pattern. This would, to be This seems like a stupendously bad idea. Warn about it in your docs, complain about it in the logs but don't willy-nilly override people's settings.
Re: Vary:User-Agent, best practices, and making the web faster.
On Sun, Jun 5, 2011 at 00:34, Joshua Marantz jmara...@google.com wrote: It was with some reluctance that I brought this up. It occurs to me that this idea propagates the sort of spec violations that led to this issue (inappropriate user of Vary:User-Agent) in the first place. However, I'm trying to figure out how to improve compliance to support legitimate uses of Vary:User-Agent without causing mod_pagespeed to become significantly less ineffective across a broad range of sites. We have found that putting complaints in Apache logs mostly causes disks to fill and servers to crash -- although that does get it noticed :). The problem, put another way, is that mod_pagespeed cannot distinguish legitimate uses of Vary:User-Agent, so it really has no business complaining in logs. Complaining in docs is fine; but some existing mod_pagespeed users that simply type sudo yum update will later notice a performance-drop and may not consult the docs to figure out why. I'm also trying to grok the first response from Eric: It's because of the other (dated) canned exceptions that set/unset no-gzip/gzip-only-text/html based on the User-Agent, to second-guess browsers that send AE:gzip but can't properly deal with it. Going backwards: which browsers send AE:gzip but can't properly deal with it? Does IE6 have that issue or is it only true of IE5? I know that IE6 has had issues with compression in the past but they appear to be addressed by patches issued by Microsoft four and a half years ago: http://support.microsoft.com/default.aspx?scid=kb;en-us;Q312496. Moreover IE6 is shrinking in market sharehttp://arstechnica.com/web/news/2011/05/web-browser-market-share-upgrade-analysis.ars(~ 10%) and IE5 does not appear in the pie-chart at all. This was indeed a (since fixed) problem with IE6. I haven't seen the gzip issue crop up since but that is purely anecdotal. And I still don't understand how that relates to Vary:User-Agent. What's really at issue here seems more related to proxies; is that right? That proxies were not respecting Accept-Encoding, but sending gzipped content to browsers that did not want it? Is that still a problem? Which proxies were broken? Are they still broken? Some popular OSS packages depend on Vary: User-Agent to make downstream proxies (reverse or forward) do the right thing. And, while I understand the reluctance to help me figure out from our module what values were passed to SetEnvIfNoCase and Header, I would like to see whether there's agreement that the Apache 2.2 docs for mod_deflate are no longer appropriate -- and in fact harmful. I've been mulling it over for 10 minutes and I can't decide. It's harmful because it leads to a proliferation of cached objects (bad) but removing it from the documentation will break things for someone somewhere (also bad).
Re: mod_gnutls and mod_proxy (TLS termination)
On Wed, May 4, 2011 at 17:50, Hardy Griech nt...@mardys.de wrote: Sorry, my fault. I focused on ssl_proxy_enable() which is not called in my case. ssl_engine_disable() does the job. So my problem is hopefully solved. Disadvantage of this solution is, that mod_ssl and mod_gnutls cannot be loaded simultaneously. I think you can work around this by chaining the optional functions. In your pre_config hook, look up and store the mod_ssl functions, then register your own. Your functions do their thing when it's mod_gnutls handling the connection and delegate to their mod_ssl counterparts otherwise.
Re: mod_gnutls and mod_proxy (TLS termination)
On Tue, May 3, 2011 at 21:10, Hardy Griech nt...@mardys.de wrote: On 03.05.2011 00:13, Ben Noordhuis wrote: On Mon, May 2, 2011 at 20:51, Hardy Griechnt...@mardys.de wrote: Now my concern is, how can I reliably catch the condition that the connection has been initiated by mod_proxy. Any ideas? r-proxyreq != PROXYREQ_NONE? Does 'initiated' mean 'request from an external reverse proxy' or 'request handled by mod_proxy'? Sorry, I forgot to mention that the code is in the pre-connection hook. So no proxyreq available :-( Also my previous patch does not work, if the destination server is on another machine. Currently I'm checking (c-sbh == NULL) to detect the mod_proxy request (yes, I meant a mod_proxy request). In mod_ssl they seem to have a similar problem with mod_proxy: mod_proxy calls some mod_ssl functions (ssl_proxy_enable() and ssl_engine_disable()) to signal a request handled by mod_proxy. I've tried to implement also these two functions - without success, they are never called also I've tried to register them just like mod_ssl does (mod_ssl is not loaded BTW). Hardy, when and where are you registering your optional functions? mod_proxy looks them up in the post_config phase so they must have been registered by then. register_hooks is a good place for it. Can you perhaps post or link to your code?
Re: mod_gnutls and mod_proxy (TLS termination)
On Fri, Apr 29, 2011 at 10:27, Hardy Griech nt...@mardys.de wrote: I'm trying to use mod_gnutls for TLS termination without success. My first suggestion would be to use mod_ssl. Alternatively, compile Apache and mod_gnutls with -g -O0 and run it with `gdb --args httpd -X -e debug`. Put a breakpoint on the pre_connection hook and take it from there.
Re: mod_gnutls and mod_proxy (TLS termination)
- mod_ssl (openssl?) does not obey the maximum fragmentation length requested by the clients I think that this has been fixed in openssl 1.0.0.a. Monkey curiosity: why do you need it? - install 'apache2-dbg' - enter gdb with the above command line - run (in gdb) - break gdb when the modules have been loaded - set the breakpoint and continue with debugging PS: anyway to automate the above procedure, esp. a breakpoint when all modules have been loaded? $ cat gdb.conf b ap_run_post_config r b your_function c $ gdb -x gdb.conf --args /path/to/httpd -X But mostly you place the breakpoint once manually, then restart the server (`r` or `run`) when you recompile your module.
Re: KeepAlive -- why is it off by default?
On Tue, Apr 12, 2011 at 02:18, Joshua Marantz jmara...@google.com wrote: What are the reasons someone might wish to turn KeepAlive off? The only one I can think of is in single-process mode (httpd -X) it can be a drag to refresh a page with lots of resources; but this seems like a secondary issue that could be worked around if needed. Lower throughput under heavy load. With KeepAlive on, each client gets allocated a process (or thread) to serve the requests. That process isn't doing anything useful between the client's requests so you end up with lots of idle processes, even when it's peak traffic. The event MPM is purposely written to deal with this issue: it puts the idle connection in a pool so the worker can serve other requests.
Re: module interaction
On Sat, Mar 26, 2011 at 16:09, Brian McQueen mcqueenor...@gmail.com wrote: If you want to share functions then put them into a library and they will be shared in the usual way like normal c functions. If you want to pass data between modules there are are notes and environment. This. And there is also the provider API and the optional function stuff in ap_provider.h and apr_optional.h.
Re: how to parse html content in handler
On Thu, Mar 24, 2011 at 13:10, Whut Jia whut_...@163.com wrote: Hi,all I want to parse a html content and withdraw some element in myself apache handler.Please ask how to do it. Thanks, Jia Hey, have a look at how mod_proxy_html[1] does it. [1] http://apache.webthing.com/mod_proxy_html/
Re: ordering output filters
On Mon, Mar 14, 2011 at 16:54, Joshua Marantz jmara...@google.com wrote: Even in the absence of 'remove_comments', it would be preferable to have mod_pagespeed run after mod_includes so that it has an opportunity to optimize the included text. The user can achieve this by putting this line into his config file: AddOutputFilter INCLUDES;MOD_PAGESPEED_OUTPUT_FILTER html But this is not desirable for a couple of reasons. We'd like to force the correct order automatically if possible. We also have a constraint that mod_pagespeed must run before mod_deflate. Actually mod_pagespeed already inserts mod_deflate in the filter-chain to run downstream of it: ap_add_output_filter(DEFLATE, NULL, request, request-connection); mod_include runs at AP_FTYPE_RESOURCE, mod_deflate at AP_FTYPE_CONTENT_SET. If you register your filter at AP_FTYPE_RESOURCE + 1 or AP_FTYPE_CONTENT_SET - 1, it will run after mod_include but before mod_deflate.
Re: Saving the original request URI ahead of a mod_rewrite
On Sun, Mar 13, 2011 at 13:15, Eric Covener cove...@gmail.com wrote: r-main doesn't change on an internal redirect AFAICT. You're right. And there is ap_internal_fast_redirect() that works different still. The only thing I can think of that should work for all three is to follow r-main until it's NULL, then follow r-prev until you're at the root request.
Re: Saving the original request URI ahead of a mod_rewrite
On Sun, Mar 13, 2011 at 04:02, Eric Covener cove...@gmail.com wrote: OP specifically mentions internal redirect and rewrite-in-htaccess. Hah, the moment I fired off that email I thought oh wait, mod_rewrite *does* do an internal redirect somewhere. Internal redirects share a pool so your suggestion would work but following r-main is still the better solution, I think, if only because it works for both redirects and sub-requests.
Re: APR global mutexes
2011/3/9 Massimo Manghi massimo.man...@unipr.it: the subject might suggest the message is an off-topic for the list. Technically it is, d...@apr.apache.org would have been a better place for it. To put it simple my question is: can every module in APR be used also to build standalone applications? More specifically: can apr_global_mutex_* calls be used this way? Yes to both (first-hand experience talking here). The very first version of this program creates a global mutex and wants to use it to synchronize access to a resource shared between parent and child processes (a shared memory area) I've found out that, when used this way, apr_global_mutex_[lock|unlock|trylock] block forever. Why? I also tried to unlock the mutex after it was created, surmising the mutex could be created by _mutex_create as locked, but also in this case apr_global_mutex_unlock blocks and never returns. Your code snippet looks sound, I can't find fault with it. You should post it to apr-dev (preferably as a complete, compilable test case), you'll get more and better feedback there. Mention on what platform and with what flags you compiled APR.
Re: Converting a 16-bit string to 8-bit?
On Sat, Mar 5, 2011 at 02:11, Adelle Hartley ade...@akemi.com.au wrote: Thanks for the feedback. Is there any documentation for the apr_xlate functions? My pleasure, Adelle. Documentation: I don't think so save for the source itself[1]. [1] https://github.com/apache/apr/blob/trunk/xlate/xlate.c
Re: Converting a 16-bit string to 8-bit?
On Fri, Mar 4, 2011 at 02:24, Adelle Hartley ade...@akemi.com.au wrote:
This is a helper class I wrote for the module I'm working on. It assumes
the native wide encoding is UTF-32. To make it cross platform, you'd have
to check what the correct wide encoding is.
This is my first apache module, so any corrections welcome.
class our_response_t
{
protected:
request_rec* m;
apr_xlate_t* m_convset;
char m_bufferBytes[CHARSET_CONVERSION_BUFFER_SIZE];
public:
our_response_t(request_rec* request) : m(request), m_convset(NULL),
m_html(NULL), m_json(NULL)
{
apr_pool_t* pool = m-pool;
apr_status_t status = apr_xlate_open(m_convset, UTF-8,
UTF-32, pool);
if (m_convset)
{
ap_set_content_type(m, text/html;charset=UTF-8);
}
}
~our_response_t()
{
if (m_convset)
{
apr_xlate_close(m_convset);
}
}
void append_chars(const wchar_t* str, size_t num_chars)
{
apr_size_t inbytes_left = num_chars*sizeof(wchar_t);
apr_size_t outbytes_left = CHARSET_CONVERSION_BUFFER_SIZE-1;
apr_status_t status = apr_xlate_conv_buffer(m_convset, (const
char*)str, inbytes_left, m_bufferBytes, outbytes_left);
m_bufferBytes[CHARSET_CONVERSION_BUFFER_SIZE-outbytes_left-1]
= 0;
ap_rputs(m_bufferBytes, m);
}
};
Adelle, your code doesn't appear to be handling errors. A number of
things can go wrong here:
1. The conversion may not be supported.
2. Partial character sequences (not an issue here since the input is
UTF-32 but I mention it for posterity's sake), reported as
APR_EINCOMPLETE.
3. Illegal character sequences, reported as APR_EINVAL.
4. Output buffer too short. Reported as APR_SUCCESS but with inbytes_left 0.
Re: Help with apr mutex
On Mon, Feb 28, 2011 at 18:26, Simone Caruso i...@simonecaruso.com wrote: I wrote a simple cache inside my module with apr_shm and apr_rmm Simone, have a look at ap_socache.h and ap_slotmem.h, they're two simple cache facilities that were added in 2.3.0. Might save you some work. :)
Re: mod_ruby on Apache for Windows 2.2.17
On Fri, Feb 11, 2011 at 08:25, Zeno Davatz zdav...@gmail.com wrote: I am trying to debug mod_ruby to load in Apache for Windows. So far Apache for Windows does start with mod_ruby.so but it seems that httpd does not start correctly with mod_ruby enabled in Apache for Windows. I don't have a solution for you but I would suggest to not do this (run mod_ruby on Windows, that is). Apache on Windows serves all requests from a single process. The Ruby interpreter is not thread-safe so mod_ruby creates a Big Mutex whenever it needs to run. Thus on Windows, with its single-process model, mod_ruby can only serve one request at a time.
Re: mod_ruby on Apache for Windows 2.2.17
On Fri, Feb 11, 2011 at 14:11, Zeno Davatz zdav...@gmail.com wrote: Apache on Windows serves all requests from a single process. Apache on Linux does not do that? Nope. The worker and event MPMs are hybrids: serving requests from many processes, where each process has many threads. And if all processes are busy, Apache will simply spin up more. The Ruby interpreter is not thread-safe so mod_ruby creates a Big Mutex whenever it needs to run. Thus on Windows, with its single-process model, mod_ruby can only serve one request at a time. So you say, that mod_ruby on Windows can _not_ leverage its power because Apache on Windows is a single thread process? Effectively single-threaded when mod_ruby is serving a request, yes.
Re: Filter to modify request headers
On Tue, Jan 25, 2011 at 23:22, Jodi Bosa jodib...@gmail.com wrote: What would be a good early hook to modify request headers that is _AFTER_ mod_ssl is finished decrypting request? When I do a ap_add_input_filter() from a ap_hook_insert_filter() seems to trigger really late (e.g. after quick_handler, post_read, etc...). post_read is a good place to tamper with headers, it's what mod_headers uses. Unconditional filters should be registered with ap_register_input_filter(), conditional filters with ap_hook_insert_filter() and ap_hook_insert_error_filter()* + ap_add_input_filter(). mod_ssl's input filter runs at AP_FTYPE_CONNECTION + 5 by the way, so you're good to go if you hook at AP_FTYPE_CONNECTION + 4. But you probably don't need to do that. * Something of a misnomer - error_filter also runs for 204 and 3xx responses.
Re: How to send a jpeg-file in Handler
2011/1/19 Whut Jia whut_...@163.com: I want to return a local jpeg-file to client when client request url is /image/metto .In handler module ,I should how to write?? ap_sub_req_lookup_uri() or ap_sub_req_lookup_file()?
Re: Help trying to figure out why an output_filter is not called.
On Wed, Jan 5, 2011 at 20:40, Joshua Marantz jmara...@google.com wrote: So if I try to remove the 'expires' filter from my handler (which runs early) then mod_expires will have a handler that runs later that inserts it after my module has completed. No, it's the other way around. mod_expires uses the insert_filter hook to insert its filter before your handler is run (and how could it be otherwise? Output filters are there to post-process the content your handler generates). Have a look at ap_invoke_handler() in config.c, that should give you a handle on how the filter chain works. But don't hesitate to post your questions if you have them, of course. :)
Re: Overriding mod_rewrite from another module
On Mon, Jan 3, 2011 at 23:19, Joshua Marantz jmara...@google.com wrote: My goal is not to remove authentication from the server; only from messing with my module's rewritten resource. The above statement is just observing that, while it's possible to shunt off mod_rewrite by returning OK from an upstream handler, the same is not true of mod_authz_host because it's invoked with a different magic macro. My bad, I parsed your post as 'mod_authz_host is a core module and cannot be removed' which is obviously false but not what you meant. Yes, all auth_checker hooks are run. You can't prevent it but you can catch the 403 on the rebound and complain loudly in the logs. Actually, that's a lie. You can prevent it and that might also answer this next bit... There may exist some buffer in Apache that's 8k. But I have traced through failing requests earlier that were more like 256 bytes. This was reported as mod_pagespeed Issue 9http://code.google.com/p/modpagespeed/issues/detail?id=9 and resolved by limiting the number of css files that could be combined together so that we did not exceed the pathname limitations. I'm pretty sure it was due to some built-in filter or core element in httpd trying to map the URL to a filename (which is not necessary as far as mod_pagespeed is concerned) and bumping into an OS path limitation (showing up as 403 Forbidden). This might be the doing of core_map_to_storage(). Never run into it myself (with URLs up to 4K, anyway) but there you go. Okay, here is a dirty secret: if you hook map_to_storage and return DONE, you bypass Apache's authentication stack - and nearly all other hooks too. Probably an exceedingly bad idea. You can however use it to prevent core_map_to_storage() from running. Just return OK and you're set. I'm still interested in your opinion on my solution where I (inspired by your hack) save the original URL in request-notes and then use *that* in my resource handler in lieu of request-unparsed_uri. This change is now committed to svn trunk (but not released in a formal patch) as http://code.google.com/p/modpagespeed/source/detail?r=348 . Sounds fine, that's the kind of stuff request notes are for.
Re: Overriding mod_rewrite from another module
On Fri, Dec 31, 2010 at 18:17, Joshua Marantz jmara...@google.com wrote: Is there a better way to solve the original problem: preventing mod_rewrite from corrupting mod_pagespeed's resources? From memory and from a quick peek at mod_rewrite.c: in your translate_name hook, set a mod_rewrite_rewritten note in r-notes with value 0 and return DECLINED. That'll trick mod_rewrite into thinking that it has already processed the request.
Re: Re: compile a file written by C++ into apache
2010/11/30 whut_jia whut_...@163.com: In Apache2.2, I compile a c++ source file with g++ as below： g++ -fPIC -shared -o mod_validate.so mod_validate.cpp -I/usr/include/httpd -I/usr/include/apr-1 -I/opt/opensaml/include After it , I copy mod_calidate.so into apache module location ,and this module work well. But now,in apache2.3,I compile this file in the same way.it accurs the following error, /apache2.3/include/http_config.h:989:error:expected ,or ... before ‘new’ (In headers file ,The 989th line is: AP_DECLARE(void) ap_merge_log_config(const struct ap_logconf *old, struct ap_logconf *new); ) I think there are not errors in this line ,but why can i compile it successfullly? new is a C++ keyword. Three solutions. 1. Rename the parameter in http_config.h to new_conf. Bad. 2. At the top of your source file add #define new new_. Bad. 3. Make your module C only. Split off the C++ code into a separate file. Good.
Re: Can i send multi-request(with Range:bytes=start-lenth) on a single connection?
2010/11/24 zhoubug chinasi...@gmail.com: Can i send multi-request(with Range:bytes=start-lenth) on a single connection? i want to reuse a connection with keep-alive,and send second request after receive the first response,but the apache response with 501 error? what should i do if i want reuse the connection? Are you sure you are on the right mailing list? This is modules-dev, maybe you're looking for users[1]? [1] http://httpd.apache.org/userslist.html
Re: ownership mmaped files - I have to be missing something...
On Thu, Nov 11, 2010 at 08:28, Mike Meyer mwm-keyword-apache.b9a...@mired.org wrote: Is there a hook that runs after config in the parent, but as the unprivileged id that I should be using? I couldn't find one (either in There isn't one, setuid() is called right before the child_init hook. Having said that, can't you open() the file in the parent and have the children mmap() the fd into memory? Just make sure it isn't marked as FD_CLOEXEC.
Re: Determining if a bucket is created by !--#include or !--#echo SSI directives
On Sat, Oct 30, 2010 at 00:24, Travis Bassetti drchong2000-apa...@yahoo.com wrote: #include directive via mod_include. Is there a way to tell when a bucket is created via #include? I want to exclude processing the bucket if it was created by the #echo directive. I can't tell if there is a difference in the bucket type when created by #include or #echo. Is there some other flag or way to differentiate these buckets? I don't think so. You could override the #echo directive with APR_RETRIEVE_OPTIONAL_FN(ap_register_include_handler) and tag the bucket in some way.
Re: How to access client socket from a protocol handler
On Sun, Oct 24, 2010 at 11:09, Alexander Farber
alexander.far...@gmail.com wrote:
So for content handlers the convention is
to use SetHandler XXX in httpd.conf and
then at the runtime they check for that string with
if (!r-handler || (strcmp(r-handler, XXX) != 0)) {
return DECLINED;
}
But for protocol handlers there is no such convention.
You have to introduce some keyword for httpd.conf
and check for it. Or in my case you could just:
if (conn-base_server-port != 843)
return DECLINED;
at the beginning? (seems to work)
Yes, that is correct. A config directive is the cleaner solution, though.
Re: How to access client socket from a protocol handler
On Sat, Oct 23, 2010 at 10:13, Alexander Farber alexander.far...@gmail.com wrote: I wonder why my mod_perl module works and the C one not. Your connection handler should return DECLINED for vhosts it doesn't handle (I wager mod_perl did this for you). You can get the vhost with conn-base_server and your module's per-server config with ap_get_module_config(conn-base_server-module_config, your_module).
Re: How to access client socket from a protocol handler
Alexander, take a look at mod_echo.c (included in the source tarball). It's a great example of how a protocol handler should work and it just might convince you to use bucket brigades after all. :) You need to check if your handler is enabled for the current vhost. If it's not, return DECLINED. If it is, look up the client socket and go from there.
Re: How to access client socket from a protocol handler
On Sun, Oct 24, 2010 at 00:00, Alexander Farber
alexander.far...@gmail.com wrote:
I've created a module using bb (the source code at the bottom)
and it suffers from the same problem (hijacks the port 80 too).
Could it be that SetHandler is a wrong directive for protocol handler?
The wrong directive, yes. SetHandler handlers work at the request
level, protocol handlers at the connection level.
Also, I do not know, how to check that the
handler is enabled for the current vhost.
From mod_echo.c:
static int process_echo_connection(conn_rec *c)
{
EchoConfig *pConfig =
ap_get_module_config(c-base_server-module_config, echo_module);
if (!pConfig-bEnabled) {
return DECLINED;
}
Hope that helps.
Re: How to access client socket from a protocol handler
On Sat, Oct 23, 2010 at 00:01, Mike Meyer mwm-keyword-apache.b9a...@mired.org wrote: I use that to get the socket so I can poll for it to have data in it, and do other things while I'm waiting. Is there a better alternative for that, or is this an exception? You could do it through apr_bucket_read(APR_NONBLOCK_READ) but polling on the socket is probably simpler, especially if you are polling on more than one fd. Just don't read or write data directly, that would bypass the filter chain (and break logging, for starters).
Re: How to access client socket from a protocol handler
On Sat, Oct 23, 2010 at 00:15, Alexander Farber alexander.far...@gmail.com wrote: Should I maybe try apr_socket_t *socket = conn-cs-desc-s or something similar instead? Probably not, the conn_config solution is most portable across Apache versions. And what do you mean by core_module in my case (source code below)? That's the reference to Apache itself, the core is a module too. Elegant, isn't it? And why is direct socket I/O bad idea, isn't this how protocol handling modules (like mod_ftp, mod_smtp) are supposed to work? There is no yes or no to this question, mostly it depends. You should strive to use what is already in place, if only because it will make your life easier down the road. Upsides to using the bucket brigade and the filter chain: * cross-platform * published and supported APIs (will work with future releases of Apache) * fairly straight-forward and transparent SSL/TLS integration Downsides: * overhead (slower) * higher learning curve So consider the pros and cons and pick the best solution. And don't hesitate to ask questions if you have them. :)
Re: How do I get hold of session information?
On Tue, Oct 19, 2010 at 17:05, Paul Donaldson pdonaldson_h...@yahoo.co.uk wrote: I assume that if I were to make a request to a web site hosted on Apache then the capability exists for one of the server side web pages to create a session and store some piece of data in it. What I want to do in my module is get hold of that session (if it exists) and read data from it. Apache core doesn't have a concept of sessions (or state as such) but take a look at mod_session.
Re: How do I get hold of session information?
On Tue, Oct 19, 2010 at 17:30, Paul Donaldson pdonaldson_h...@yahoo.co.uk wrote: Thank you. I will take a look at mod_session. Will my module be able to check if mod_session is enabled (sorry, I don't know the Apache terminology) and, if it is, talk to it and ask it for what it has stored in its session? Yes. It exports an environment variable, see the SessionEnv directive for details.
Re: Memory Pool
Martin, if you are working in a constrained environment, then you are probably better off using something like libmicrohttpd[1] or libevent's evhttp interface[2]. Apache has a rather heavy resource footprint. [1] http://www.gnu.org/software/libmicrohttpd/ [2] http://monkey.org/~provos/libevent/doxygen/evhttp_8h.html
Re: Memory Pool
On Mon, Oct 11, 2010 at 16:14, Martin Townsend martin.towns...@power-oasis.com wrote: I have created a pool from the child pool for storing warning messages that can live across requests, the final request will insert the warnings into the response. How do I ensure that this pool is cleared at the end of the final request? By calling apr_pool_clear() or apr_pool_destroy()?
Re: Memory Pool
On Mon, Oct 11, 2010 at 16:40, Martin Townsend martin.towns...@power-oasis.com wrote: use, or should I set a flag and then use a hook like fix-ups that will check this flag and then call apr_pool_clear()? This. You can use a request note for a flag.
Re: New module for anonymous ip logging
Hi Franz, welcome. Replies inline: On Wed, Oct 6, 2010 at 00:49, Franz Schwartau fr...@electromail.org wrote: How should the module react to a failed initialization of seed_rand() in iphash_create_server_config() (line 90)? Returning NULL in iphash_create_server_config() doesn't seem to help. I'd like to disable the module somehow if the random generator could be initialized properly. Add a 'initialized' flag to iphash_config_t and check its value in log_ip_hash. That said, if security is an issue - which I presume it is - you are probably better off aborting. No other module checks if the return value of apr_palloc() or apr_pcalloc() is NULL. Does it mean memory allocation via apr_palloc() will never fail and memory size is indefinitely? ;-) Yep. And if memory turns out to be finite after all, the APR will tear down the process.
Re: determine if mod called via ssl request
On Tue, Sep 21, 2010 at 17:24, J.D. Mullin jeremym1...@cableone.net wrote:
It seems like I am missing some very simple way to tell if the request was
made via https, but I have scanned all of the structures available from the
request_rec and I don't see anything obvious.
const char *flag = getenv(HTTPS);
if (flag !strcmp(flag, on)) {
// HTTPS-only logic
}
Re: add cookie in handler process
2010/9/19 whut_jia whut_...@163.com: The question is that I just get Location header but get Cookie header when I access to apache server.why?? Not sure what the question is. Do you mean that the response contains a Location header but no Set-Cookie header? If so, you probably need to use apr_table_add() instead of apr_table_set().
Re: modules architecture issue
Fabio, do you need a module for this? It sounds like it better belongs in an application server like Tomcat.
Re: ap_sub_req_lookup_file vs ap_sub_req_lookup_uri
Peter, what version of Apache are you testing this with?
Re: ap_sub_req_lookup_file vs ap_sub_req_lookup_uri
ap_sub_req_lookup_file() should work, it's what mod_include uses when you have !--#include file=/foo/bar-- on your page. You might want to take a look at its source.
Re: Problem with ap_internal_redirect_handler
On Tue, Aug 24, 2010 at 11:11, Martin Townsend martin.towns...@power-oasis.com wrote: I'm trying to add a captcha feature to our custom module. How should I handle the returned captcha code in the POST data. I currently have an input filter for processing other POST data but if I use this how do I redirect to the captcha.shtml page if validation fails. If I validate the code in an earlier phase like the fixups how do I read the POST data? Does anyone know the best way to handle this. I'm using v2.2-3. Martin, can't your filter set a request note that one of your hooks further down the line picks up?
Re: Problem with ap_internal_redirect_handler
On Tue, Aug 24, 2010 at 11:50, Martin Townsend martin.towns...@power-oasis.com wrote: Do you mean setting a request note from the input filter that an output filter picks up which can then output the captcha.shtml? Yes, if your module consists of filters only. If it also includes a content generator, I'd probably do it from there. But you know your code best so pick the most appropriate solution.
Re: Problem with ap_internal_redirect_handler
On Tue, Aug 24, 2010 at 14:56, Ray Morris supp...@bettercgi.com wrote: This sounds to me like an access_checker, which should be called from the access_checker hook. Filters, as I understand it, are for transforming the content - compressing it, for example. Entirely replacing the content with an unrelated document sounds like an abuse of filters. I wonder if this might be a case of if all you have is a hammer ... Agreed, but I don't think the request body is available when the access_checker hook is run (the OP mentioned he needed to extract POST data).
Re: opensaml unmarshall Exception
You should post your question to the OpenSAML mailing list, this isn't the place for it.
Re: Re: a questions about calling opensaml lib
You didn't mention what MPM you are using but I assume it's the prefork one. You need to register a hook with ap_hook_child_init() and call SAMLConfig::getConfig().init() from there.
Re: a questions about calling opensaml lib
Is this with the prefork or threaded MPM? And what is the SAML library are you using? Shibboleth's?
Re: Can an Apache module inject configuration in runtime?
On Tue, Jun 1, 2010 at 11:39, Andrew Godziuk and...@cloudaccess.net wrote: What I want to achieve is injecting new vhosts without Apache restart. Of course I'm aware that the changes would fully take effect after all workers have recycled, but for me - it's still better than a restart. Andrew, wouldn't you be better off with something like mod_vhost_alias, as-is or as leitmotiv? Propagating configuration changes to all workers is rather difficult, especially with pre-fork MPMs.
Re: Error in apr_hash.
Jaysingh, could you provide more details about the errors (error messages?) you get?
Re: HTTP Streaming with Apache 1
I wonder if it's possible with Apache 2? To get hold of the client socket, so that I can poll() or select() it and implement server push aka Comet r-connection-cs-pfd.desc.s-socketdes if I'm not mistaken. Check out apr_pollset_create() if you are going to do polling inside Apache or sendmsg(2) if you want to pass the fd to another process.
Re: Weird SSL issue with module
On Mon, Sep 21, 2009 at 23:14, Joe Nardone jnard...@gmail.com wrote: Under 2.2, it appears that no matter what I do, the headers and data are being sent in two separate SSL records in the response. This is true at least for POST requests. How large are these POST requests, Joe? I ask because SSL_read() doesn't support records 16 kB (which may or may not be the maximum size for TLS records - it probably is, but don't take my word for it). http://www.openssl.org/docs/ssl/SSL_read.html
Re: Output filter order selection
On Mon, Sep 14, 2009 at 21:39, Ivan Ristic ivan.ris...@gmail.com wrote:
There's an incompatibility between ModSecurity and mod_deflate, which
I would like to fix it. (It is triggered when AddOutputFilterByType is
used.) I basically need to ensure that ModSecurity's output filter
runs before mod_deflate's in all cases. I am aware of mod_filter
(which I suspect should be able to deal with this situation), but I
prefer a solution that does not require further work on the part of
ModSecurity users.
It depends. If module X registers its output filter in, for example,
the post-config hook, you can register your own hook and have it run
before module X's like so:
const char *runAfterUs = { mod_x.c, NULL };
ap_hook_post_config(your_post_config_hook, NULL, runAfterUs, HOOK_MIDDLE);
If however the module registers its output filter during the 'register
hooks' phase like mod_deflate does, your best bet is to register your
own filter and manipulate the ap_filter_rec_t structure (which is
essentially a linked list) so that your filter comes before
mod_deflate's.
It's not entirely according to the book but it should be a fairly
robust solution. ap_filter_rec_t lives in util_filter, which is part
of the module API, and isn't a opaque structure so I don't foresee it
changing anytime soon.
This is, of course, to the best of my knowledge. I'm not aware of a
better way, but if there is, I'd also like to hear it. =)
Re: mod_authn_dbd - internal server error after certain idle time
On 7/7/07, Nick Kew [EMAIL PROTECTED] wrote: Any other suggestions for a workaround to check the connection now? With MySQL 5.0.3 and later, mysql_ping() doesn't automatically reconnect but returns CR_SERVER_GONE_ERROR instead, so you'll want to reinitialize the prepared statements when that happens. I'm not aware of a reliable workaround for earlier versions of MySQL, unfortunately.
Re: mod_authn_dbd - internal server error after certain idle time
On 7/7/07, Ben Noordhuis [EMAIL PROTECTED] wrote:
I'm not aware of a reliable workaround for earlier versions of
MySQL, unfortunately.
Allow me to reply to myself. If you can live with a slight performance
impact, the code snippet below should help you to reconnect in a
controlled fashion. Hope this helps.
switch (mysql_query(mysql, SELECT 1)) {
case 0:
mysql_free_result(mysql_use_result(mysql));
break;
case CR_SERVER_GONE_ERROR:
/* reinitialize */
break;
default:
/* handle error */
}
