Hi, Same opinion and all the other load testing tools (Gatling, LoadRunner, Silkperformer, etc.) have it
Antonio 2015-02-10 9:56 GMT+01:00 <[email protected]>: > As a user I think this feature would be very helpful. > > Reports are a JMeter weakest feature, so enhancements in this area are > very welcome. > > It would also be nice if it as easy as possible to use. Imho this means > that it should be added to JMeter and not to a separate commandline utility > etc. > > Regards, > Pascal > > -----Ursprüngliche Nachricht----- > Von: Felix Schumacher [mailto:[email protected]] > Gesendet: Montag, 9. Februar 2015 18:55 > An: [email protected] > Betreff: Re: New Listener to generate HTML report at end of load test Was > APDEX Computing and reporting > > Am 03.02.2015 um 13:38 schrieb Philippe Mouawad: > > Hi, > > Any additional opinions of dev team or users ? > > I 'd like to know if I should continue efforts on this or stop it. > I think it would be a nice feature to have a simple report at the end of a > jmeter run. Maybe the user should ask for it by specifying a command line > parameter like "--report". > > If this is integrated in the shell wrapper it could even be a program that > is started after the jmeter run. But I don't think users will generally > start another program themselves, so we shouldn't hide it too deep. > > Regards > Felix > > > > Thanks > > Regards > > > > On Monday, February 2, 2015, Philippe Mouawad > > <[email protected]> > > wrote: > > > >> > >> On Monday, February 2, 2015, sebb <[email protected] > >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> > >>> On 1 February 2015 at 22:41, Philippe Mouawad > >>> <[email protected]> wrote: > >>>> On Sunday, February 1, 2015, sebb <[email protected]> wrote: > >>>> > >>>>> On 1 February 2015 at 19:31, Philippe Mouawad > >>>>> <[email protected] <javascript:;>> wrote: > >>>>>> Hi , > >>>>>> I uploaded a screenshot and new patch code showing what I meant. > >>>>>> I think including it in core should be considered seriously. > >>>>> If the code reads the CSV file at the end of a test to produce the > >>>>> HTML file, there is no need for it to be included in the JMeter > >>>>> code which is used to run tests. > >>>>> It should be a separate application. > >>>>> > >>>>> I don't share your opinion sebb. > >>>> Having a report ready at end of load test is nowadays a standard, > >>>> look > >>> at > >>>> market tools wether open source or not. > >>>> I don't see the problem here, can you give more details ? > >>> The problem is that it adds extra code to the JMeter application > >>> which takes extra memory. > >> > >> My last implementation only uses memory at end of test not morz during > it. > >> > >> > >>> Also, if the user wants to process existing files, it will be easier > >> to have a standalone application rather than having to start the > >>> JMeter test application and then find the reporting tool. > >> > >> It is possible with it, just make a plan with 1 debug sampler and put > >> listener. Report will be generated. > >> > >>> It would be useful if the tool could be used in batch mode to > >>> process a set of CSV files. > >> > >> Maybe, but I think it's an additional feature. Also having another > >> tool may lead to what happened to the report package, it seems it was > >> never used. But if you implement it it will be ok for me. > >> > >> > >>> It's also likely to be easier for others to contribute additional > >>> reporting tools if they are part of a separate application. > >> I don't understand this ? how is it easier if it's in jmeter core. If > >> it's apart I don't support this option. > >> > >> > >> > >>> It should make the coding requirements simpler. > >> how ? > >> > >>>> Having another application( by the way do you mean provided as a > >>>> tool in jmeter or third party) means in the first case you need to > >>>> setup another tool and in the latter case need to develop your own, > or use or pay one. > >>> Of course the code should be included with JMeter. > >>> But it should be a separate tool. > >>> > >>>> This is the major drawback of JMeter I hear from customers and read > >>> around > >>>> the net. > >>>> I first though it would not be that easy but it appears we are able > >>>> to reuse existing code. > >>> I don't follow. > >>> > >>>> If you disagree, I think we should wait for other team members > >>>> their opinion and/or setup a vote for this. > >>> The main JMeter code should be reserved for setting up and running > tests. > >>> > >> IMHO reporting must be part of JMeter, and anyway it is already > >> through some Listeners , which although not perfect bring useful > >> infos. The aim of this new listener is: > >> 1/ Allow generation in gui and non gui mode 2/ Have a "sexy" (not yet > >> ) report readable in browser with dynamic behaviour (zoom, select > >> some samples...) 3/ Allow generation at end of load test 4/ Make it > >> easily customizable as FTL will be a property 5/ Make it extensible > >> in the future > >> > >> > >> > >> > >>>> > >>>> > >>>> > >>>>>> @Felix thanks for review of initial patch. > >>>>>> Regards > >>>>>> > >>>>>> On Sunday, February 1, 2015, sebb <[email protected] > >>>>>> <javascript:;>> > >>>>> wrote: > >>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad > >>>>>>> <[email protected] <javascript:;> <javascript:;>> wrote: > >>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher < > >>>>>>>> [email protected] <javascript:;> > >>>>>>>> <javascript:;>> > >>>>> wrote: > >>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad: > >>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> I intend to commit a BackendListener client implementation > >>>>>>>>>> that > >>>>>>> computes > >>>>>>>>>> APDEX at end of Load Test. > >>>>>>>>>> It will take: > >>>>>>>>>> > >>>>>>>>>> - Acceptable Response Time (http://www.apdex.fr/) T > >>>>>>>>>> > >>>>>>>>> The link was in french (my french is not very good (really non > >>>>> existant > >>>>>>>>> :)), but luckily wikipedia had an article about apdex, which I > >>> could > >>>>>>> read. > >>>>>>>> Sorry for the french link some references: > >>>>>>>> http://www.apdex.org/overview.html > >>>>>>>> > >>>>>>>> - Compute F as 4xT but allow customization > >>>>>>>>>> - List of samples taken into account > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> This listener will compute it during load test and generate > >>>>>>>>>> an > >>> HTML > >>>>>>> file > >>>>>>>>>> at > >>>>>>>>>> end of test containing it. > >>>>>>>>>> > >>>>>>>>>> This listener could be later enhanced to : > >>>>>>>>>> > >>>>>>>>>> - Generate equivalent of Aggregate Report > >>>>>>>>>> - Possibly graphs based on chartjs or some other > >>>>>>>>>> graphing JS > >>>>>>> Library > >>>>>>>>> I understand, that apdex is a simple mean to have one metric. > >>>>>>>>> As > >>>>> such I > >>>>>>>>> wonder why we would want to graph it. > >>>>>>>> In fact my proposal was larger than just APDEX. > >>>>>>>> What I am proposing is to implement a report generator that at > >>> end of > >>>>>>> Test > >>>>>>>> would compute HTML page with: > >>>>>>>> - APDEX > >>>>>>>> - Some graphs like: > >>>>>>>> - Plot of OK/KO > >>>>>>>> (http://www.chartjs.org/docs/#doughnut-pie-chart > >>> ) > >>>>>>>> - Request Summary (what's currently in Aggregate Graph) > >>>>>>>> http://www.chartjs.org/docs/#bar-chart > >>>>>>>> - Response Time over time ( > >>> http://www.chartjs.org/docs/#line-chart) > >>>>>>>> I think all this should in fact be computed at end of load test > >>>>> instead > >>>>>>> of > >>>>>>>> during it. > >>>>>>>> The input of this listener would be the generated CSV file, and > >>>>>>>> at > >>>>> end of > >>>>>>>> Test it would read it an compute all this. > >>>>>>> In which case it should be a stand-alone app. > >>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> If we want to display a graph of apdex values for different > >>> tolerated > >>>>>>>>> times, than we would have to store all values, which would > >>>>>>>>> make > >>> the > >>>>>>>>> listener quite heavy on the memory side. > >>>>>>>>> > >>>>>>>>>> Thoughts ? > >>>>>>>>>> > >>>>>>>>> How will errors be counted? Sometimes I can cope with errors, > >>>>>>>>> as > >>>>> long as > >>>>>>>>> they are reported fast and sometimes an error would result in > >>>>>>>>> a > >>> very > >>>>>>>>> unhappy consumer. > >>>>>>>>> > >>>>>>>> Good catch, I think only successful samples must be taken into > >>>>> account. > >>>>>>>> Error will count only toward total requests (unsatisfied). > >>>>>>>> > >>>>>>>>> Regards > >>>>>>>>> Felix > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Cordialement. > >>>>>>>> Philippe Mouawad. > >>>>>> > >>>>>> -- > >>>>>> Cordialement. > >>>>>> Philippe Mouawad. > >>>> > >>>> -- > >>>> Cordialement. > >>>> Philippe Mouawad. > >> > >> -- > >> Cordialement. > >> Philippe Mouawad. > >> > >> > >> > >> > >
