On Mon, Sep 10, 2012 at 3:53 PM, Alex Bligh <a...@alex.org.uk> wrote: > Ben, > > >> No, but the documentation omits some crucial details. >> apr_pool_create() is thread-safe only if: >> >> 1. libapr is compiled with APR_HAS_THREADS >> 2. APR_POOL_DEBUG is turned off >> 3. the parent pool has a thread-safe allocator (which is true for the >> global allocator that is used when parent=NULL, provided conditions #1 >> and #2 are satisfied) >> >> The pools you get from httpd core satisfy #3 but a module may replace >> e.g. r->pool with another pool that doesn't. Ergo, don't rely on a >> pool being thread safe unless you explicitly make it so. > > > Ah, OK. I had thought that was only an issue as when the pool create > was running (at which point I am single threaded). But I can fix > that - thanks. > > >>>> That won't solve all your problems though. Bucket brigades are not >>>> thread safe, you will need something to synchronize on. >>> >>> >>> >>> So what I was trying to do was to use >>> a) the input bucket brigade in thread #1 (main thread) >>> b) the output bucket brigade in thread #2 >>> in an attempt to avoid synchronization >>> >>> But what I don't understand is whether thread #2, in writing >>> to the output filters (which presumably have a reference >>> to r->pool) will need synchronisation. >> >> >> Yes. It's not just because r->pool may or may not be synchronized, the >> internal structure of the bucket brigade is not protected by any locks >> either. > > > Oh I understand that, but I thought in the example above only > thread #1 would be accessing the input bucket brigade and > only thread #2 would be accessing the output bucket brigade, > so there would be no need for synchronisation as they were > thread local. > > >>> And if I have to synchronize, how do I do that in practice? >>> Thread #2 does and ap_fwrite/ap_flush so I can hold a mutex >>> there. But what do I do in thread #1, which calls ap_brigade_get >>> and blocks? I can't hold a mutex during that. I can make it >>> a non-blocking ap_brigade_get (if I understood how to do it) >> >> >> Non-blocking reads are pretty straightforward: >> >> apr_thread_mutex_lock(&mutex); >> rv = ap_get_brigade(f->next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ, >> len); if (APR_STATUS_IS_EAGAIN(rv)) apr_thread_cond_wait(&cond, &mutex); >> rv = ap_get_brigade(f->next, bb, AP_MODE_READBYTES, APR_NONBLOCK_READ, >> len); apr_thread_mutex_unlock(&mutex); >> >> The other thread wakes up this thread with apr_thread_cond_signal(&cond). > > > I think I may not have explained what I am doing clearly. One thread > is doing input (from the apache client), and the other output (to > the apache client). The ap_brigade_get is in the input thread (the > main thread) and is blocking on the client sending more data. So > the thing that would need to wake the thread up is more data becoming > ready from the client - nothing to do with the other > thread. I don't know how to detect that. > > >>> but what I really need is the equivalent of a select() which >>> I can do with the mutex not held (or some way to drop the mutex >>> during the raw reads). Any ideas? >> >> >> You could set up a pollset in the main thread and funnel incoming data >> into your bucket brigade. Not terribly efficient (lots of context >> switches) but the real world impact may very well be negligible and >> you can support multi-process setups with zero changes to your code. > > > Hmm, well if I could use a pollset in my main thread I wouldn't > need bucket brigades at all. But as this is data coming from the > client, surely it's going to be in a bucket brigade already as > it will have passed through all the input filters etc., having > been read by apache itself? > > Diagramatically: > > > | APACHE | ........... MODULE ........... | > > Client <==> Apache ----> ap_get_brigade ----> do_something [thread1] > ^ > |------ ap_fwrite <--- do_something_else [thread2] > > Each thread has a different bucket brigade with a different allocator. > > ap_get_brigade either needs to block, or if it's non-blocking it > needs to wait on a condwait or something for /apache/ to produce > more data from the client, not on the other thread.
Right, I think I see what you mean. Apache may not be a perfect fit. I once had to solve a similar issue and I eventually settled on sending the socket to another process. Managing mostly dormant connections turned out to be too much of a headache to do from within httpd.
