On 14 September 2014 15:04, Philippe Mouawad <[email protected]> wrote: > On Sun, Sep 14, 2014 at 3:22 PM, sebb <[email protected]> wrote: > >> On 1 September 2014 20:25, Philippe Mouawad <[email protected]> >> wrote: >> > Hello, >> > Working on: >> > - https://issues.apache.org/bugzilla/show_bug.cgi?id=52073 >> > I am currently stucked. >> > >> > To recap the initial issue: >> > - To download embedded resources we init and shutdown a >> ThreadPoolExecutor >> > limiting the benefit of a Thread pool to the sole Sampler call. >> > - As a side effect (see below) , as this will result in the init of a new >> > HttpClient this will not perform fine due to SSLContext eager creation >> (see >> > below) >> > >> > So I started working on a patch that: >> > - Made the ThreadPoolExecutor static >> > - Used a Blocking Queue to limit the number of downloads 1 Sampler can do >> > >> > See the attached patch. >> > >> > And this is were I hit a problem. >> > Indeed we hold in ThreadLocal the HttpClient instance per Thread which >> each >> > thread needs to cleanup at end. >> > But with this design when thread ends it cleans up its state, this is a >> > problem: >> > - Currently an HttpClient will be reinitialized for each thread of the >> > Pool, this init has a cost particularly due to eager init of SSL context >> , >> > see >> > >> http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201405.mbox/%3C1400143180.23146.7.camel@ubuntu%3E >> > - For the issue, this is a bigger problem, as I need to hold the list of >> > HttpSamplerProxy to call threadFinished at end, but this cannot work as >> the >> > list would grow huge accros the lifetime of a thread during a load test >> > ending with OOM. It grows huge because each time we call AsyncSample#call >> > we clone the HTTPSamplerBase and register it in the list. Furthermore we >> > cannot call "threadFinished" at end of Callable call (AsynSample) >> > >> > >> > Thinking about the fix I have the following ideas but they require some >> > work: >> > - First rework HC usage to only use 1 HttpClient for all threads >> >> Also need to move to latest HC version. >> I made a start on that, but found some test issues involving POST data >> formatting that I have yet to resolve. >> >> Great ! > >> - Rework ThreadLocal usage: >> > * Currently we use ThreadLocal 47 times but the use is split among 2 Use >> > case: >> > 1) One is to improve performance, this one looks OK to me >> > 2) One is in fact a way to associate User Data to Thread , the thread >> > being in fact a User. I think we should rework those Thread Locals and >> move >> > them to a singleton that would be keyed on userId (could be a simple >> > increased number associated to thread). Doing so would allow us in the >> > future to move from Threads to a ThreadPool. >> >> I think we need to distinguish JMeter threads and Java threads here. >> JMeter threads correspond to Users. >> > We're on the same line > >> So one way to deal with this issue would be to associate some data >> with the JMeterThread class instances. >> Either by using the class instance as the key, or by adding data to >> the class itself. >> > I think by key it would be better, maybe create a class dedicated to hold > User data. > We would put this in Thread Local when Task (of ThreadPoolExecutor) is > assigned to a Thread and remove it from Thread Local on Task end > using ThreadPoolExecutor#beforeExecute and ThreadPoolExecutor#afterExecute
One cannot use standard Java ThreadLocal for data that is shared between Java threads. > >> We would need to be careful to ensure access is thread-safe as the >> data might be used by multiple Java threads. >> Also need to be careful to clean up the data when the JMeterThread >> finishes. >> >> > Thoughts ? >> > >> > Regards >> > Philippe >> > @philmdot >> > > > > -- > Cordialement. > Philippe Mouawad.
