On Sun, Sep 14, 2014 at 3:22 PM, sebb <[email protected]> wrote: > On 1 September 2014 20:25, Philippe Mouawad <[email protected]> > wrote: > > Hello, > > Working on: > > - https://issues.apache.org/bugzilla/show_bug.cgi?id=52073 > > I am currently stucked. > > > > To recap the initial issue: > > - To download embedded resources we init and shutdown a > ThreadPoolExecutor > > limiting the benefit of a Thread pool to the sole Sampler call. > > - As a side effect (see below) , as this will result in the init of a new > > HttpClient this will not perform fine due to SSLContext eager creation > (see > > below) > > > > So I started working on a patch that: > > - Made the ThreadPoolExecutor static > > - Used a Blocking Queue to limit the number of downloads 1 Sampler can do > > > > See the attached patch. > > > > And this is were I hit a problem. > > Indeed we hold in ThreadLocal the HttpClient instance per Thread which > each > > thread needs to cleanup at end. > > But with this design when thread ends it cleans up its state, this is a > > problem: > > - Currently an HttpClient will be reinitialized for each thread of the > > Pool, this init has a cost particularly due to eager init of SSL context > , > > see > > > http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201405.mbox/%3C1400143180.23146.7.camel@ubuntu%3E > > - For the issue, this is a bigger problem, as I need to hold the list of > > HttpSamplerProxy to call threadFinished at end, but this cannot work as > the > > list would grow huge accros the lifetime of a thread during a load test > > ending with OOM. It grows huge because each time we call AsyncSample#call > > we clone the HTTPSamplerBase and register it in the list. Furthermore we > > cannot call "threadFinished" at end of Callable call (AsynSample) > > > > > > Thinking about the fix I have the following ideas but they require some > > work: > > - First rework HC usage to only use 1 HttpClient for all threads > > Also need to move to latest HC version. > I made a start on that, but found some test issues involving POST data > formatting that I have yet to resolve. > > Great !
> - Rework ThreadLocal usage: > > * Currently we use ThreadLocal 47 times but the use is split among 2 Use > > case: > > 1) One is to improve performance, this one looks OK to me > > 2) One is in fact a way to associate User Data to Thread , the thread > > being in fact a User. I think we should rework those Thread Locals and > move > > them to a singleton that would be keyed on userId (could be a simple > > increased number associated to thread). Doing so would allow us in the > > future to move from Threads to a ThreadPool. > > I think we need to distinguish JMeter threads and Java threads here. > JMeter threads correspond to Users. > We're on the same line > So one way to deal with this issue would be to associate some data > with the JMeterThread class instances. > Either by using the class instance as the key, or by adding data to > the class itself. > I think by key it would be better, maybe create a class dedicated to hold User data. We would put this in Thread Local when Task (of ThreadPoolExecutor) is assigned to a Thread and remove it from Thread Local on Task end using ThreadPoolExecutor#beforeExecute and ThreadPoolExecutor#afterExecute > We would need to be careful to ensure access is thread-safe as the > data might be used by multiple Java threads. > Also need to be careful to clean up the data when the JMeterThread > finishes. > > > Thoughts ? > > > > Regards > > Philippe > > @philmdot > -- Cordialement. Philippe Mouawad.
