Date: 2004-12-14T17:18:39
Editor: SonamChauhan <[EMAIL PROTECTED]>
Wiki: Apache JMeter Wiki
Page: LogAnalysis
URL: http://wiki.apache.org/jakarta-jmeter/LogAnalysis
Explain Perl parser a bit better, especially regarding HTTP redirects
Change Log:
------------------------------------------------------------------------------
@@ -1,29 +1,17 @@
== Suggestions and Recipes for Log Analysis ==
Please add your tips and tricks to this page. Extraction scripts and Excel
Macros etc welcome!
-
-
-Here's how Sonam imports and process JMeter log data in Excel:
----
-1. First, I have to get the data into Excel. I use Perl to parse the XML logs
and generate a delimited file Excel imports.
+Here are the steps Sonam Chauhan uses to import and process JMeter log data in
Excel. Steps #3 and 4 are quite painstaking. It would be nice to have a macro
to automate those steps - however, it is beyond my Excel abilities at the
moment.:
-The heart of my Perl data-mangler is this regular expression:
-{{{
-# Each line in logfile is matched to this regular expression to grab relevant
details from the log entry.
+1. First, generate delimited file for import into Excel.
+I use Perl to parse the XML logs and generate a delimited file for import into
Excel. The heart of the Perl script is the regular expression below. Each line
in the JMeter logfile must be matched to this expression:
+{{{
+# Regular expression extracts relevant fields from the log entry.
/timeStamp="(\d+)".+?threadName="(.*?)".+?label="(.+?)"
time="(\d+?)".+?success="(.+?)"/;
-# Explanation for regex:
-# A normal log entry line logs one HTTP 'sampler' operation:
-# <sampleResult timeStamp="..." ... threadName="..." label="..."
time="..." />
-# In case the opertion had HTTP redirects, the redirects show up as
-# nested <sampleResult> elements, but still on the same line:
-# <sampleResult timeStamp= ... > <sampleResult timeStamp=.... > ...
</sampleResult>
-# We are only interested in the data in the first <sampleResult> element,
-# so we use non-greedy pattern match operator ( '.*?' or '.+?') to ignore
-# any latter <sampleResult> elements,
-
-# The temporary Perl variables this regular expression generates are used to
write a comma-delimited line.
+# Data in the regex variables is accessed for use in building
suitably-delimited line
my $timestamp = $1; # unix timestamp
my $threadname = $2; # thread label
my $label = $3; # operation label
@@ -31,14 +19,37 @@
my $success = $5; # boolean success indicator for this operation
}}}
-2. Once the data is in Excel, I convert the timestamp column from Jmeter's
Unix timestamp format (base year 1970) to the Excel format (base year 1900)
using this following formula. This formula is applied to the entire timestamp
column. (NOTE: a better formula was mentioned on JMeter users list sometime
back)
+The complexity of the regular expression is required to parse nested log
entries that occur during HTTP redirects. A normal log entry line logs one HTTP
'sampler' operation. For eg:
+{{{
+<sampleResult ... time="156" ... />
+}}}
+
+However, when the opertion had a HTTP redirect (HTTP status code 302), JMeter
records the redirects as nested {{{<sampleResult>}}} elements -- these which
still occur on the same line as the log entry:
+{{{<sampleResult ... time="2750"> <sampleResult ... time="2468" ... />
<sampleResult time="141" .../> <sampleResult ... time="141" .../>
</sampleResult>
+}}}
+
+The outermost {{{<sampleResult>}}} element has time = 2750 milliseconds. This
is the sum of times of the nested redirect elements. We are only interested in
the data contained in the outermost element. Hence the regular expression uses
the non-greedy pattern match operator ( {{{.*?}}} or {{{.+?}}}) to ignore the
latter {{{<sampleResult>}}} elements.
+
+On the other hand, Excel 2002 can directly import JMeter XML format logs.
However, it has problems with entries for HTTP 302 redirects. The nested log
entry example above will generate three rows in Excel, with the total time
repeated thrice. i.e:
+{{{
+Login 2750
+Login 2750
+Login 2750
+}}}
+
+
+2. Convert Timestamps to Excel Format
+
+Once the data is in Excel, I convert the timestamp column from Jmeter's Unix
timestamp format (base year 1970) to the Excel format (base year 1900) using
this following formula. This formula is applied to the entire timestamp column.
(NOTE: a better formula was mentioned on JMeter users list sometime back)
{{{
=(x/1000+((365*70+17)*86400))/86400
}}}
-3. I then sort rows are sorted by operation name (i.e. JMeter sampler name)
-4. I can now generate suitable reports. For instance, I generate a graph of
page load times v/s time for different operations (e.g.: login, add 1 line to
the order, etc). A different series for each operation type is used.
+3. Now sort rows on the operation name (i.e. JMeter sampler name)
+
+4. Generate suitable reports and graphs manually.
+
+For instance, one can generate a graph of page load times v/s time for
different operations (e.g.: login, add 1 line to the order, etc). A different
series in the graph is needed for each operation type used - this can be quite
painstaking to add to a graph when there is a lot of data.
-Steps #3 and 4 are quite painstaking. It would be nice to have a macro to
automate those steps - however, it is beyond my Excel abilities at the moment.
----
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
