Ben,
No, but the documentation omits some crucial details.
apr_pool_create() is thread-safe only if:
1. libapr is compiled with APR_HAS_THREADS
2. APR_POOL_DEBUG is turned off
3. the parent pool has a thread-safe allocator (which is true for the
global allocator that is used when parent=NULL, provided conditions #1
and #2 are satisfied)
The pools you get from httpd core satisfy #3 but a module may replace
e.g. r->pool with another pool that doesn't. Ergo, don't rely on a
pool being thread safe unless you explicitly make it so.
Ah, OK. I had thought that was only an issue as when the pool create
was running (at which point I am single threaded). But I can fix
that - thanks.
That won't solve all your problems though. Bucket brigades are not
thread safe, you will need something to synchronize on.
So what I was trying to do was to use
a) the input bucket brigade in thread #1 (main thread)
b) the output bucket brigade in thread #2
in an attempt to avoid synchronization
But what I don't understand is whether thread #2, in writing
to the output filters (which presumably have a reference
to r->pool) will need synchronisation.
Yes. It's not just because r->pool may or may not be synchronized, the
internal structure of the bucket brigade is not protected by any locks
either.
Oh I understand that, but I thought in the example above only
thread #1 would be accessing the input bucket brigade and
only thread #2 would be accessing the output bucket brigade,
so there would be no need for synchronisation as they were
thread local.
And if I have to synchronize, how do I do that in practice?
Thread #2 does and ap_fwrite/ap_flush so I can hold a mutex
there. But what do I do in thread #1, which calls ap_brigade_get
and blocks? I can't hold a mutex during that. I can make it
a non-blocking ap_brigade_get (if I understood how to do it)
Non-blocking reads are pretty straightforward:
apr_thread_mutex_lock(&mutex);
rv = ap_get_brigade(f->next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ,
len); if (APR_STATUS_IS_EAGAIN(rv)) apr_thread_cond_wait(&cond, &mutex);
rv = ap_get_brigade(f->next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ,
len); apr_thread_mutex_unlock(&mutex);
The other thread wakes up this thread with apr_thread_cond_signal(&cond).
I think I may not have explained what I am doing clearly. One thread
is doing input (from the apache client), and the other output (to
the apache client). The ap_brigade_get is in the input thread (the
main thread) and is blocking on the client sending more data. So
the thing that would need to wake the thread up is more data becoming
ready from the client - nothing to do with the other
thread. I don't know how to detect that.
but what I really need is the equivalent of a select() which
I can do with the mutex not held (or some way to drop the mutex
during the raw reads). Any ideas?
You could set up a pollset in the main thread and funnel incoming data
into your bucket brigade. Not terribly efficient (lots of context
switches) but the real world impact may very well be negligible and
you can support multi-process setups with zero changes to your code.
Hmm, well if I could use a pollset in my main thread I wouldn't
need bucket brigades at all. But as this is data coming from the
client, surely it's going to be in a bucket brigade already as
it will have passed through all the input filters etc., having
been read by apache itself?
Diagramatically:
| APACHE | ........... MODULE ........... |
Client <==> Apache ----> ap_get_brigade ----> do_something [thread1]
^
|------ ap_fwrite <--- do_something_else [thread2]
Each thread has a different bucket brigade with a different allocator.
ap_get_brigade either needs to block, or if it's non-blocking it
needs to wait on a condwait or something for /apache/ to produce
more data from the client, not on the other thread.
--
Alex Bligh
