Re: Very long tests with huge JTL log file

[Adrian Speteanu]

This is what I always wanted JMeter to do out of the box, if it supports
log4j or slf4j, so that you don't always have long setup times before you
start a test (there are tons of other things you have to consider, except
how often do you need the files to be splitted).

For that matter, does anyone know of a Linux / Unix utility that can help
with this? (might also be useful to learn to have available for other apps
as well you know).

On Thu, Sep 8, 2011 at 11:43 PM, Tonimenen  wrote:

> The idea was doing some kind of rotatelog in JTL, and analyze old files
> generated with other jmeter in gui mode, and after analyzing droping old JTL
> files.
>
> Enviado desde mi iPad
>
> El 08/09/2011, a las 20:21, Shay Ginsbourg 
> escribió:
>
> > Fine.
> > Thanks.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >*Shay Ginsbourg*
> > Regulatory & Testing Affairs Consultant
> >
> >  Formerly QA Manager of LoadRunner at Mercury Interactive
> >
> >  M.Sc. cum laude in Bio-Medical Engineering
> >  M.Sc. in Mechanical Engineering
> >
> >
> >
> >
> >
> > *Work:* 035185873
> > *Mobile:* 0546690915
> >
> > *Email:* sginsbo...@gmail.com
> >
> > *http://il.linkedin.com/in/shayginsbourg*
> >
> > *GINSBOURG.COM* 
> >--
> >
> >
> >
> > *P** **Please consider your environmental responsibility before printing
> > this e-mail*.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Sep 8, 2011 at 8:55 PM, sebb  wrote:
> >
> >> On 8 September 2011 18:49, Shay Ginsbourg  wrote:
> >>> Adding a summary line (to jmeter.log) is an interesting option.
> >>> How is the "Summariser" specified in a script?
> >>>
> >>
> >>
> >>
> http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results
> >>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>   *Shay Ginsbourg*
> >>> Regulatory & Testing Affairs Consultant
> >>>
> >>> Formerly QA Manager of LoadRunner at Mercury Interactive
> >>>
> >>> M.Sc. cum laude in Bio-Medical Engineering
> >>> M.Sc. in Mechanical Engineering
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> *Work:* 035185873
> >>> *Mobile:* 0546690915
> >>>
> >>> *Email:* sginsbo...@gmail.com
> >>>
> >>> *http://il.linkedin.com/in/shayginsbourg*
> >>>
> >>> *GINSBOURG.COM* 
> >>>   --
> >>>
> >>>
> >>>
> >>> *P** **Please consider your environmental responsibility before
> printing
> >>> this e-mail*.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Sep 8, 2011 at 8:46 PM, sebb  wrote:
> >>>
>  If you want to get a rough idea of the test performance, you can add a
>  Summariser which will log a summary line (to jmeter.log) every so
>  often - 3 mins by default.
> 
>  This works also in non-GUI mode.
> 
>  On 8 September 2011 18:35, Deepak Shetty  wrote:
> >> DO you know anyway to rotate the log JTL file ?
> > ditto to what everyone else is saying - you dont want to do this.
> > just load it up into your favorite rdbms or OLAP tool and analyse
> from
> > there.
> >
> > On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <
> >> tonime...@gmail.com
> > wrote:
> >
> >> Hello,
> >>
> >> I am going to execute a tests for 3 days, it will made me to have a
> >> very
> >> long JTL file. DO you know anyway to rotate the log JTL file ?
> >>
> >> Toni.
> >>
> >
> 
>  -
>  To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
>  For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
> 
> 
> >>>
> >>
> >> -
> >> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
> >> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
> >>
> >>
>
> -
> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
>
>

Re: Very long tests with huge JTL log file

[Tonimenen]

The idea was doing some kind of rotatelog in JTL, and analyze old files 
generated with other jmeter in gui mode, and after analyzing droping old JTL 
files.

Enviado desde mi iPad

El 08/09/2011, a las 20:21, Shay Ginsbourg  escribió:

> Fine.
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
>*Shay Ginsbourg*
> Regulatory & Testing Affairs Consultant
> 
>  Formerly QA Manager of LoadRunner at Mercury Interactive
> 
>  M.Sc. cum laude in Bio-Medical Engineering
>  M.Sc. in Mechanical Engineering
> 
> 
> 
> 
> 
> *Work:* 035185873
> *Mobile:* 0546690915
> 
> *Email:* sginsbo...@gmail.com
> 
> *http://il.linkedin.com/in/shayginsbourg*
> 
> *GINSBOURG.COM* 
>--
> 
> 
> 
> *P** **Please consider your environmental responsibility before printing
> this e-mail*.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Thu, Sep 8, 2011 at 8:55 PM, sebb  wrote:
> 
>> On 8 September 2011 18:49, Shay Ginsbourg  wrote:
>>> Adding a summary line (to jmeter.log) is an interesting option.
>>> How is the "Summariser" specified in a script?
>>> 
>> 
>> 
>> http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results
>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>>   *Shay Ginsbourg*
>>> Regulatory & Testing Affairs Consultant
>>> 
>>> Formerly QA Manager of LoadRunner at Mercury Interactive
>>> 
>>> M.Sc. cum laude in Bio-Medical Engineering
>>> M.Sc. in Mechanical Engineering
>>> 
>>> 
>>> 
>>> 
>>> 
>>> *Work:* 035185873
>>> *Mobile:* 0546690915
>>> 
>>> *Email:* sginsbo...@gmail.com
>>> 
>>> *http://il.linkedin.com/in/shayginsbourg*
>>> 
>>> *GINSBOURG.COM* 
>>>   --
>>> 
>>> 
>>> 
>>> *P** **Please consider your environmental responsibility before printing
>>> this e-mail*.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Thu, Sep 8, 2011 at 8:46 PM, sebb  wrote:
>>> 
 If you want to get a rough idea of the test performance, you can add a
 Summariser which will log a summary line (to jmeter.log) every so
 often - 3 mins by default.
 
 This works also in non-GUI mode.
 
 On 8 September 2011 18:35, Deepak Shetty  wrote:
>> DO you know anyway to rotate the log JTL file ?
> ditto to what everyone else is saying - you dont want to do this.
> just load it up into your favorite rdbms or OLAP tool and analyse from
> there.
> 
> On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <
>> tonime...@gmail.com
> wrote:
> 
>> Hello,
>> 
>> I am going to execute a tests for 3 days, it will made me to have a
>> very
>> long JTL file. DO you know anyway to rotate the log JTL file ?
>> 
>> Toni.
>> 
> 
 
 -
 To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
 For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
 
 
>>> 
>> 
>> -
>> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
>> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
>> 
>> 

-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org

Re: Very long tests with huge JTL log file

[Shay Ginsbourg]

Fine.
Thanks.









*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant

  Formerly QA Manager of LoadRunner at Mercury Interactive

  M.Sc. cum laude in Bio-Medical Engineering
  M.Sc. in Mechanical Engineering





*Work:* 035185873
*Mobile:* 0546690915

*Email:* sginsbo...@gmail.com

*http://il.linkedin.com/in/shayginsbourg*

*GINSBOURG.COM* 
--



*P** **Please consider your environmental responsibility before printing
this e-mail*.
















On Thu, Sep 8, 2011 at 8:55 PM, sebb  wrote:

> On 8 September 2011 18:49, Shay Ginsbourg  wrote:
> > Adding a summary line (to jmeter.log) is an interesting option.
> > How is the "Summariser" specified in a script?
> >
>
>
> http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results
>
> >
> >
> >
> >
> >
> >
> >
> >*Shay Ginsbourg*
> > Regulatory & Testing Affairs Consultant
> >
> >  Formerly QA Manager of LoadRunner at Mercury Interactive
> >
> >  M.Sc. cum laude in Bio-Medical Engineering
> >  M.Sc. in Mechanical Engineering
> >
> >
> >
> >
> >
> > *Work:* 035185873
> > *Mobile:* 0546690915
> >
> > *Email:* sginsbo...@gmail.com
> >
> > *http://il.linkedin.com/in/shayginsbourg*
> >
> > *GINSBOURG.COM* 
> >--
> >
> >
> >
> > *P** **Please consider your environmental responsibility before printing
> > this e-mail*.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Sep 8, 2011 at 8:46 PM, sebb  wrote:
> >
> >> If you want to get a rough idea of the test performance, you can add a
> >> Summariser which will log a summary line (to jmeter.log) every so
> >> often - 3 mins by default.
> >>
> >> This works also in non-GUI mode.
> >>
> >> On 8 September 2011 18:35, Deepak Shetty  wrote:
> >> >> DO you know anyway to rotate the log JTL file ?
> >> > ditto to what everyone else is saying - you dont want to do this.
> >> > just load it up into your favorite rdbms or OLAP tool and analyse from
> >> > there.
> >> >
> >> > On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <
> tonime...@gmail.com
> >> >wrote:
> >> >
> >> >> Hello,
> >> >>
> >> >> I am going to execute a tests for 3 days, it will made me to have a
> very
> >> >> long JTL file. DO you know anyway to rotate the log JTL file ?
> >> >>
> >> >> Toni.
> >> >>
> >> >
> >>
> >> -
> >> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
> >> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
> >>
> >>
> >
>
> -
> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
>
>

Re: Very long tests with huge JTL log file

[sebb]

On 8 September 2011 18:49, Shay Ginsbourg  wrote:
> Adding a summary line (to jmeter.log) is an interesting option.
> How is the "Summariser" specified in a script?
>

http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results

>
>
>
>
>
>
>
>    *Shay Ginsbourg*
> Regulatory & Testing Affairs Consultant
>
>  Formerly QA Manager of LoadRunner at Mercury Interactive
>
>  M.Sc. cum laude in Bio-Medical Engineering
>  M.Sc. in Mechanical Engineering
>
>
>
>
>
> *Work:* 035185873
> *Mobile:* 0546690915
>
> *Email:* sginsbo...@gmail.com
>
> *http://il.linkedin.com/in/shayginsbourg*
>
> *GINSBOURG.COM* 
>    --
>
>
>
> *P** **Please consider your environmental responsibility before printing
> this e-mail*.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Sep 8, 2011 at 8:46 PM, sebb  wrote:
>
>> If you want to get a rough idea of the test performance, you can add a
>> Summariser which will log a summary line (to jmeter.log) every so
>> often - 3 mins by default.
>>
>> This works also in non-GUI mode.
>>
>> On 8 September 2011 18:35, Deepak Shetty  wrote:
>> >> DO you know anyway to rotate the log JTL file ?
>> > ditto to what everyone else is saying - you dont want to do this.
>> > just load it up into your favorite rdbms or OLAP tool and analyse from
>> > there.
>> >
>> > On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez > >wrote:
>> >
>> >> Hello,
>> >>
>> >> I am going to execute a tests for 3 days, it will made me to have a very
>> >> long JTL file. DO you know anyway to rotate the log JTL file ?
>> >>
>> >> Toni.
>> >>
>> >
>>
>> -
>> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
>> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
>>
>>
>

-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org

Re: Very long tests with huge JTL log file

[Shay Ginsbourg]

Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?









*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant

  Formerly QA Manager of LoadRunner at Mercury Interactive

  M.Sc. cum laude in Bio-Medical Engineering
  M.Sc. in Mechanical Engineering





*Work:* 035185873
*Mobile:* 0546690915

*Email:* sginsbo...@gmail.com

*http://il.linkedin.com/in/shayginsbourg*

*GINSBOURG.COM* 
--



*P** **Please consider your environmental responsibility before printing
this e-mail*.
















On Thu, Sep 8, 2011 at 8:46 PM, sebb  wrote:

> If you want to get a rough idea of the test performance, you can add a
> Summariser which will log a summary line (to jmeter.log) every so
> often - 3 mins by default.
>
> This works also in non-GUI mode.
>
> On 8 September 2011 18:35, Deepak Shetty  wrote:
> >> DO you know anyway to rotate the log JTL file ?
> > ditto to what everyone else is saying - you dont want to do this.
> > just load it up into your favorite rdbms or OLAP tool and analyse from
> > there.
> >
> > On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez  >wrote:
> >
> >> Hello,
> >>
> >> I am going to execute a tests for 3 days, it will made me to have a very
> >> long JTL file. DO you know anyway to rotate the log JTL file ?
> >>
> >> Toni.
> >>
> >
>
> -
> To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
>
>

Re: Very long tests with huge JTL log file

[sebb]

If you want to get a rough idea of the test performance, you can add a
Summariser which will log a summary line (to jmeter.log) every so
often - 3 mins by default.

This works also in non-GUI mode.

On 8 September 2011 18:35, Deepak Shetty  wrote:
>> DO you know anyway to rotate the log JTL file ?
> ditto to what everyone else is saying - you dont want to do this.
> just load it up into your favorite rdbms or OLAP tool and analyse from
> there.
>
> On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez 
> wrote:
>
>> Hello,
>>
>> I am going to execute a tests for 3 days, it will made me to have a very
>> long JTL file. DO you know anyway to rotate the log JTL file ?
>>
>> Toni.
>>
>

-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org

Re: Very long tests with huge JTL log file

[Deepak Shetty]

> DO you know anyway to rotate the log JTL file ?
ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.

On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez wrote:

> Hello,
>
> I am going to execute a tests for 3 days, it will made me to have a very
> long JTL file. DO you know anyway to rotate the log JTL file ?
>
> Toni.
>

Re: Very long tests with huge JTL log file

[Oliver Lloyd]

I've run tests like this resulting in jtl files greater than 25GB, I used the
method mentioned by Nermin, running over 20 servers each running a test that
was configured to execute only 5% of my target load. Works absolutely fine.
(Note. Amazon is your friend for such shenanigans.)

Obviously: use CSV mode, don't log responses, run on the command line.

If you end up with data this big then you really need to be aggregating the
files in situe before trying to download them and I didn't bother trying to
view them locally in the JM GUI either. Instead I loaded the data into mySQL
and played with it from there. There's some pretty cool open source ETL
tools to help with this. 

Also, we've recently started playing with mongo for super big datasets -
man, it is fast.

-
http://www.http503.com/
--
View this message in context: 
http://jmeter.512774.n5.nabble.com/Very-long-tests-with-huge-JTL-log-file-tp4783083p4783329.html
Sent from the JMeter - User mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org

Re: Very long tests with huge JTL log file

[Nermin CALUK]

First ideas that come to my mind:
- split your test into, say, 5 consecutive threads, each one doing 20% of your 
test and writing into a separate file OR experiment with a nested loop to 
achieve this (you can use timestamp variable as a part of the name of your 
file, or you can use counter if you use loops)
- select less options to log in Configuration
- use CSV instead of XML logging, if possible
- write ERRORS in one file, SUCCESSES in another file 


From my experience, huge files won't be a problem if you run JMeter from 
command line (and if you have sufficient disk space of course :)


Cheers, 
Nermin


- Original Message -
On 06:26 PM 09/08/2011 Toni Menendez Lopez wrote:

Hello,

I am going to execute a tests for 3 days, it will made me to have a very

long JTL file. DO you know anyway to rotate the log JTL file ?

Toni.



-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org

Re: Very long tests with huge JTL log file

[apc]

I know none. 

But what the problem with huge files? Why you don't like them?

-
--
Andrey Pohilko
JP@GC Maintainer
--
View this message in context: 
http://jmeter.512774.n5.nabble.com/Very-long-tests-with-huge-JTL-log-file-tp4783083p4783110.html
Sent from the JMeter - User mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org