Hi, For your information, APM tools use APDEX (and a little more) in UEM features And customers like it (easy to make some great dashboard) And I think it will be great to have it in JMeter
Antonio 2015-02-03 13:38 GMT+01:00 Philippe Mouawad <[email protected]>: > Hi, > Any additional opinions of dev team or users ? > I 'd like to know if I should continue efforts on this or stop it. > > Thanks > Regards > > On Monday, February 2, 2015, Philippe Mouawad <[email protected]> > wrote: > > > > > > > On Monday, February 2, 2015, sebb <[email protected] > > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > > >> On 1 February 2015 at 22:41, Philippe Mouawad > >> <[email protected]> wrote: > >> > On Sunday, February 1, 2015, sebb <[email protected]> wrote: > >> > > >> >> On 1 February 2015 at 19:31, Philippe Mouawad > >> >> <[email protected] <javascript:;>> wrote: > >> >> > Hi , > >> >> > I uploaded a screenshot and new patch code showing what I meant. > >> >> > I think including it in core should be considered seriously. > >> >> > >> >> If the code reads the CSV file at the end of a test to produce the > >> >> HTML file, there is no need for it to be included in the JMeter code > >> >> which is used to run tests. > >> >> It should be a separate application. > >> >> > >> >> I don't share your opinion sebb. > >> > Having a report ready at end of load test is nowadays a standard, look > >> at > >> > market tools wether open source or not. > >> > I don't see the problem here, can you give more details ? > >> > >> The problem is that it adds extra code to the JMeter application which > >> takes extra memory. > > > > > > My last implementation only uses memory at end of test not morz during > it. > > > > > >> Also, if the user wants to process existing files, it will be easier > > > > to have a standalone application rather than having to start the > >> JMeter test application and then find the reporting tool. > > > > > > It is possible with it, just make a plan with 1 debug sampler and put > > listener. Report will be generated. > > > >> > >> It would be useful if the tool could be used in batch mode to process > >> a set of CSV files. > > > > > > Maybe, but I think it's an additional feature. Also having another tool > > may lead to what happened to the report package, it seems it was never > > used. But if you implement it it will be ok for me. > > > > > >> > >> It's also likely to be easier for others to contribute additional > >> reporting tools if they are part of a separate application. > > > > I don't understand this ? how is it easier if it's in jmeter core. If > it's > > apart I don't support this option. > > > > > > > >> It should make the coding requirements simpler. > > > > how ? > > > >> > >> > Having another application( by the way do you mean provided as a tool > in > >> > jmeter or third party) means in the first case you need to setup > another > >> > tool and in the latter case need to develop your own, or use or pay > one. > >> > >> Of course the code should be included with JMeter. > >> But it should be a separate tool. > >> > >> > This is the major drawback of JMeter I hear from customers and read > >> around > >> > the net. > >> > >> > I first though it would not be that easy but it appears we are able to > >> > reuse existing code. > >> > >> I don't follow. > >> > >> > If you disagree, I think we should wait for other team members their > >> > opinion and/or setup a vote for this. > >> > >> The main JMeter code should be reserved for setting up and running > tests. > >> > > IMHO reporting must be part of JMeter, and anyway it is already through > > some Listeners , which although not perfect bring useful infos. The aim > of > > this new listener is: > > 1/ Allow generation in gui and non gui mode > > 2/ Have a "sexy" (not yet ) report readable in browser with dynamic > > behaviour (zoom, select some samples...) > > 3/ Allow generation at end of load test > > 4/ Make it easily customizable as FTL will be a property > > 5/ Make it extensible in the future > > > > > > > > > >> > > >> > > >> > > >> > > >> >> > @Felix thanks for review of initial patch. > >> >> > Regards > >> >> > > >> >> > On Sunday, February 1, 2015, sebb <[email protected] > <javascript:;>> > >> >> wrote: > >> >> > > >> >> >> On 31 January 2015 at 13:16, Philippe Mouawad > >> >> >> <[email protected] <javascript:;> <javascript:;>> wrote: > >> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher < > >> >> >> > [email protected] <javascript:;> > <javascript:;>> > >> >> wrote: > >> >> >> > > >> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad: > >> >> >> >> > >> >> >> >>> Hi, > >> >> >> >>> I intend to commit a BackendListener client implementation > that > >> >> >> computes > >> >> >> >>> APDEX at end of Load Test. > >> >> >> >>> It will take: > >> >> >> >>> > >> >> >> >>> - Acceptable Response Time (http://www.apdex.fr/) T > >> >> >> >>> > >> >> >> >> The link was in french (my french is not very good (really non > >> >> existant > >> >> >> >> :)), but luckily wikipedia had an article about apdex, which I > >> could > >> >> >> read. > >> >> >> >> > >> >> >> > > >> >> >> > Sorry for the french link some references: > >> >> >> > http://www.apdex.org/overview.html > >> >> >> > > >> >> >> > - Compute F as 4xT but allow customization > >> >> >> >>> - List of samples taken into account > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> This listener will compute it during load test and generate an > >> HTML > >> >> >> file > >> >> >> >>> at > >> >> >> >>> end of test containing it. > >> >> >> >>> > >> >> >> >>> This listener could be later enhanced to : > >> >> >> >>> > >> >> >> >>> - Generate equivalent of Aggregate Report > >> >> >> >>> - Possibly graphs based on chartjs or some other graphing > JS > >> >> >> Library > >> >> >> >>> > >> >> >> >> I understand, that apdex is a simple mean to have one metric. > As > >> >> such I > >> >> >> >> wonder why we would want to graph it. > >> >> >> > > >> >> >> > In fact my proposal was larger than just APDEX. > >> >> >> > What I am proposing is to implement a report generator that at > >> end of > >> >> >> Test > >> >> >> > would compute HTML page with: > >> >> >> > - APDEX > >> >> >> > - Some graphs like: > >> >> >> > - Plot of OK/KO ( > http://www.chartjs.org/docs/#doughnut-pie-chart > >> ) > >> >> >> > - Request Summary (what's currently in Aggregate Graph) > >> >> >> > http://www.chartjs.org/docs/#bar-chart > >> >> >> > - Response Time over time ( > >> http://www.chartjs.org/docs/#line-chart) > >> >> >> > > >> >> >> > I think all this should in fact be computed at end of load test > >> >> instead > >> >> >> of > >> >> >> > during it. > >> >> >> > The input of this listener would be the generated CSV file, and > at > >> >> end of > >> >> >> > Test it would read it an compute all this. > >> >> >> > >> >> >> In which case it should be a stand-alone app. > >> >> >> > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> >> If we want to display a graph of apdex values for different > >> tolerated > >> >> >> >> times, than we would have to store all values, which would make > >> the > >> >> >> >> listener quite heavy on the memory side. > >> >> >> >> > >> >> >> >>> > >> >> >> >>> Thoughts ? > >> >> >> >>> > >> >> >> >> How will errors be counted? Sometimes I can cope with errors, > as > >> >> long as > >> >> >> >> they are reported fast and sometimes an error would result in a > >> very > >> >> >> >> unhappy consumer. > >> >> >> >> > >> >> >> > Good catch, I think only successful samples must be taken into > >> >> account. > >> >> >> > Error will count only toward total requests (unsatisfied). > >> >> >> > > >> >> >> >> > >> >> >> >> Regards > >> >> >> >> Felix > >> >> >> >> > >> >> >> >> > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > Cordialement. > >> >> >> > Philippe Mouawad. > >> >> >> > >> >> > > >> >> > > >> >> > -- > >> >> > Cordialement. > >> >> > Philippe Mouawad. > >> >> > >> > > >> > > >> > -- > >> > Cordialement. > >> > Philippe Mouawad. > >> > > > > > > -- > > Cordialement. > > Philippe Mouawad. > > > > > > > > > > -- > Cordialement. > Philippe Mouawad. >
