So to be clear: that's simply detecting errors that rise above a certain threshold. But currently, there's no way to track performance degradation over time, correct?
What I want is an automatic way to spot degradation job-over-job, such that Jenkins would realize "Your tests are now 10% slower than they were a week ago". Or is that asking for too much, and perhaps for trouble? On Tue, Jul 16, 2013 at 10:05 AM, Cedrick Johnson < [email protected]> wrote: > This is contained in Jenkins. I don't know about Maven and setting that > up. We are using the standard JMeter plugin within Jenkins. If you activate > it that should work. Here's my Build step in Jenkins (Execute Shell and > yeah, we're still on 2.8) > > rm -f *.jtl > $HOME/apache-jmeter-2.8/bin/jmeter -n -t SomeTestPlan.jmx -JServerName= > wee.com -JServerPort=8080 -JUserThreads=50 -JUserLoopCount=1 -l > RhubarbTestResults.jtl > > That works for us, and has caught some pretty big design changes that > slowed things down. > > -c > > -----Original Message----- > From: Shmuel Krakower [mailto:[email protected]] > Sent: Tuesday, July 16, 2013 1:29 AM > To: JMeter Users List > Subject: RE: Load testing, Continuous Integration, failing on > build-over-build degradation > > Hi Cedrick > Thanks for sharing but is this post build action is part of the maven > plugin or part of jenkins? > > I am looking for exactly this capability for couple of months now! Can you > point on any link to brief introduction of this as I couldn't find any. > On Jul 15, 2013 10:01 PM, "Cedrick Johnson" < > [email protected]> > wrote: > > > When you configure your JMeter Jenkins job, in Post-Build actions you > > can have it publish the performance test result report which points to > > the Test Results .jtl file that is generated when running the test. In > > that report, there's a Performance Threshold section where you can set > > it to identify when the build is unstable (number of errors exceeds > > this percentage > > amount) or build Failed when the number of errors exceeds this set > amount. > > > > The errors are determined in your actual load test, i.e. if requests > > start timing out, or other conditions that you are checking in your > > tests begin failing they will count against this threshold and Jenkins > > will alert you to a degradation in performance once those errors are met. > > > > -c > > > > -----Original Message----- > > From: Shmuel Krakower [mailto:[email protected]] > > Sent: Monday, July 15, 2013 1:54 PM > > To: JMeter Users List > > Subject: Re: Load testing, Continuous Integration, failing on > > build-over-build degradation > > > > Hi Adrian > > Thanks for sharing but how exactly u control the response times > > thresholds or error rates? > > I cannot find any control of this... > > On Jul 15, 2013 4:26 PM, "Adrian Speteanu" <[email protected]> wrote: > > > > > Hi, > > > > > > Check my attempt of an answer bellow. > > > > > > Regards, > > > Adrian S > > > > > > On Mon, Jul 15, 2013 at 2:56 PM, Marc Esher <[email protected]> > > wrote: > > > > > > > Greetings all, > > > > > > > > I'm integrating our load tests into our CI environment, with the > > > > goal of identifying performance degradation as soon as possible. > > > > The idea is is > > > to > > > > use some kind of threshold, from one CI build to the next, to > > > > identify > > > when > > > > performance has dipped to an unacceptable level from one run to > > another. > > > > > > > > I'm using Jenkins, currently. > > > > > > > > Anyone have any guidance, strategy, experience, wisdom here? > > > > > > > > The Jenkins Performance Plugin is decent for reporting trends, but > > > > it has no capabilities to automatically spot problems. > > > > > > > > > > What is your exact expectation regarding to this last phrase? > > > > > > I'm currently using the maven plugin, and it integrates nicely with > > > the jenkins plugin that you mentioned. The tests fail when expected. > > > Here are the configurations made to the pom.xml (I followed the > > > tutorial from the jenkins plugin project when first setting up this > > > test project). The threshold for failures are set in the jenkins > > > plugin > > and they work. > > > > > > <groupId>com.lazerycode.jmeter</groupId> > > > <artifactId>jmeter-maven-plugin</artifactId> > > > ... > > > <executions> > > > <execution> > > > <id>jmeter-tests</id> > > > <phase>verify</phase> > > > <goals> > > > <goal>jmeter</goal> > > > </goals> > > > </execution> > > > </executions> > > > > > > execution: #mvn clean verify > > > > > > > > > > Thanks! > > > > > > > > Marc > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > >
