Interesting discussion. mod_http2 had similar issues to overcome and there should be plenty of room for improvements...
> Am 27.01.2016 um 20:50 schrieb Ruediger Pluem <[email protected]>: > On 01/27/2016 12:46 PM, Yann Ylavic wrote: >> On Wed, Jan 27, 2016 at 10:02 AM, Plüm, Rüdiger, Vodafone Group >> <[email protected]> wrote: >>> >>>> -----Original Message----- >>>> From: Yann Ylavic [mailto:[email protected]] >>>> Sent: Mittwoch, 27. Januar 2016 09:15 >>>> To: httpd-dev >>>> Subject: Re: svn commit: r1726787 - >>>> /httpd/httpd/trunk/modules/proxy/mod_proxy_wstunnel.c >>>> >>>> I'm not sure the buckets can be destroyed while in flight in >>>> mod_proxy_wstunnel, unlike with mod_proxy_http where the backend >>>> connection/request may be cleared before all data are out. >>>> >>>> ISTM that this can't happen here because we flush out on every write >>> >>> In theory this should be the case, but I think having brigade and buckets >>> allocated from >>> the same bucket allocator is a "better safe than sorry" approach that >>> catches possible >>> edge case now or in the future (think of Grahams ideas to have suspending >>> filters). >>> The issue with these possible edge cases is that it will be hard to find >>> the root cause, >>> because we likely segfault somewhere else. > > Just saw that the code for mod_proxy_connect is very similar to the one from > mod_proxy_wstunnel > and thus could impose the same issues. Maybe it should be fixed as well. Just for my understanding of ap_buckets_lifetime_transform(): - duplicates buckets from one brigade+allocator to another - optimizes the case where source buckets live at least as long as the copied ones - can therefore use transient buckets and avoids buffer copies mod_http2: - does not make that lifetime assumption. needs to copy. - has optimization for transferring file buckets by setaside'ing the apr_file to the target pool Maybe both approaches could benefit from each other. What are the tasks to synchronize: A. reading from somewhere into a source brigade B. transferring from source to another target brigade C. writing the target brigade to somewhere else where A and C keep the current brigade API. Maybe we could introduce something like a "Transfer Window" that keeps track of target buckets and their destruction which then triggers - maybe at another point in time / another thread - the destruction of the corresponding source buckets. There should be a maximum memory footprint for the "transfer window" which suspends further transfers until target buckets have been destroyed. The transfer window could be given mutex and cond for working multi-threaded. Ideally, this would be chainable, so that we have more than two brigades involved. -Stefan
