On Fri, Feb 6, 2015 at 9:32 PM, Vladimir Sitnikov < [email protected]> wrote:
> > Did you try Constant Throughput Timer > > I did, however it does have drawbacks. > 1.1) It does not always work. Sorry, it was long ago I've last tested > it, so I cannot list lots of examples from the top of my head. > For instance, I've just created 100 threads, set thread duration to be > 20 seconds and put constant throughput timer of 60 per minute. > I expect exactly 20 items of "debug sampler" to appear, but 100 happens. > Test plan: > ThreadGroup(100 threads, 20 seconds test duration) > Loop(Forever) > DebugSampler(for_throughput) > ConstantThroughputTimer(60 per minute) > DebugSampler(work) > I suggest you open bugs with if possible jmx plan showing issue. > > 1.2) It tries to adapt to the target based on the duration of the loop > execution. > It works really awful for long iterations. For instance, if a single > iteration takes 10 minutes and I want 10 per hour, then CTT has just > no time to "adapt". > If an iteration takes longer than CTT expected, then you're screwed. > You get less executions than required. > > 1.3) CTT is not reproducible. See point 3 of original email on > "Repeatable test profile". > > 1.4) CTT tries to do even delay between iterations. This plays awful > when you have multiple scenarios running at once if the scenarios have > the same target throughput. You end up with sin(t) kind of CPU%. > > 5) "Bursty load without explicit batches". If I set CTT at 1 per > minute, it will never launch iterations closer than 60 seconds > (assuming all the bugs like 1.1 are fixed). However, in reality > arrivals often have spikes. For instance, in poisson process (those > kind of arrivals are common to happen in telecom industry) > inter-arrival durations vary (bingo!). > In other words, if an action takes 30 seconds to execute and CTT > launches it every 60 seconds, then this kind of testing will NEVER > catch a case when two actions run concurrently. With inter-arrival > delays like 10 sec, 110 sec, 80 sec, 40 sec throughput would still > result in "1 per minute" on average, but it will catch more > performance bugs (e.g. the one when two concurrent actions result in > out of memory). > > > Yes you can, just schedule them : > > Scheduler > Startup Delay + Duration > > It does not scale: suppose you have 20 thread groups (e.g. 20 UI > scenarios). > How would you manage all those durations? > How would you change the duration if throughput needs to be changed? > (hint: in scalability tests throughput is often need to be altered for > each run) > How would you pass the script to your buddy? > Okay. Did you try this one from 3rd party: http://jmeter-plugins.org/wiki/ThroughputShapingTimer/ > > Even if you could configure it once, those 0:0:0 defaults will > definitely bite you very soon (as a new group is added). > > Vladimir > Thanks for this feedback, it will help us improve. Regards
