> Am 21.10.2015 um 16:48 schrieb Graham Leggett <minf...@sharp.fm>: > > On 21 Oct 2015, at 4:18 PM, Stefan Eissing <stefan.eiss...@greenbytes.de> > wrote: >> 6. pool buckets are very tricky to optimize, as pool creation/destroy is not >> thread-safe in general >> and it depends on how the parent pools and their allocators are set up. >> Early hopes get easily crushed under load. > > As soon as I see “buckets aren’t thread safe” I read it as “buckets are being > misused” or “pool lifetimes are being mixed up". > > Buckets arise from allocators, and you must never try add a bucket from one > allocator into a brigade sourced from another allocator. For example, if you > have a bucket allocated from the slave connection, you need to copy it into a > different bucket allocated from the master connection before trying to add it > to a master brigade. > > Buckets are also allocated from pools, and pools have different lifetimes > depending on what they were created for. If you allocate a bucket from the > request pool, that bucket will vanish when the request pool is destroyed. > Buckets can be passed from one pool to another, that is what “setaside” means. > > It is really important to get the pool lifetimes right. Allocate something > accidentally from the master connection pool on a slave connection and it > appears to work, because generally the master outlives the slave. Until the > master is cleaned up first, and suddenly memory vanishes unexpectedly in the > slave connections - and you crash. > > There were a number of subtle bugs in the proxy where buckets had been > allocated from the wrong pool, and all sorts of weirdness ensued. Make sure > your pool lifetimes are allocated correctly and it will work.
This is all true and correct - as long as all this happens in a single thread. If you have multiple threads and create sub pools for each from a main pool, each and every create and destroy of these sub-pools, plus any action on the main pool must be mutex protected. I found out. Similar with buckets. When you create a bucket in one thread, you may not destroy it in another - *while* the bucket_allocator is being used. bucket_allocators are not thread-safe, which means bucket_brigades are not, which means that all buckets from the same brigade must only be used inside a single thread. This means for example that, even though mod_http2 manages the pool lifetime correctly, it cannot pass a response bucket from a request pool in thread A for writing onto the main connection in thread B, *as long as* the response is not complete and thread A is still producing more buckets with the same allocator. etc. etc. That is what I mean with not-thread-safe. //Stefan
