Re: [Xen-devel] [PATCH 09/27] ts-unixbench-reslts: process and plot bench results
On Wed, Dec 10, 2014 at 07:10:06PM +0100, Dario Faggioli wrote:
From: Dario Faggioli raist...@linux.it
Mangle the results of a run of unixbench a bit, so that
they can be plotted. This also produces a (gnu)plot script
and the plot itself. All is saved in $stash, for the
running flight and job.
This is done in a new Osstest/Benchmarking.pm module, as
the functions introduced may turn out useful somewhere else
too.
I would suggest using a dedicated commit for the introduction of
Benchmarking.pm.
The results are read from the original unixbench results
file and a new file, with basically a table in it is
produced. Gnuplot uses such file as data for the plotting.
A gnuplot script is produced and invoked (and saved in $stash)
rather than using the gnuplot perl binding because, this way,
one can modify the plotting commands a bit, and regen the
plot(s).
Signed-off-by: Dario Faggioli dario.faggi...@citrix.com
Cc: Wei Liu wei.l...@citrix.com
Cc: Ian Campbell ian.campb...@citrix.com
Cc: Ian Jackson ian.jack...@eu.citrix.com
---
Osstest/Benchmarking.pm | 115
+++
ts-unixbench-reslts | 17 +++
2 files changed, 132 insertions(+)
create mode 100644 Osstest/Benchmarking.pm
diff --git a/Osstest/Benchmarking.pm b/Osstest/Benchmarking.pm
new file mode 100644
index 000..0c5c538
--- /dev/null
+++ b/Osstest/Benchmarking.pm
@@ -0,0 +1,115 @@
+# This is part of osstest, an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
2009-2014
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+#
+
+package Osstest::Benchmarking;
+
+use strict;
+use warnings;
+
+use IO::File;
+use IPC::Cmd qw[can_run run];
+
+use Osstest;
+use Osstest::TestSupport;
+
+BEGIN {
+use Exporter ();
+our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+$VERSION = 1.00;
+@ISA = qw(Exporter);
+@EXPORT = qw(unixbench_process_results
+ unixbench_print_results
+ unixbench_plot_results
+ );
+%EXPORT_TAGS = ( );
+
+@EXPORT_OK = qw();
+}
+
+#-- manipulation of benchmarks results --
+
+sub unixbench_process_results ($$) {
+ my ($results_ref,$rfilen)= @_;
+ my $h= new IO::File $rfilen or die $!;
+
+ my $par;
+ while ($h) {
+my ($bench,$val,$idx);
+if (m/.*running ([0-9]*) parallel.*$/) {
+ $par= $1;
+}
+if
(m/^(\S[a-zA-z0-9-\(\)\s]*)\s([0-9]+.[0-9]?)\s*([0-9]+.[0-9]?)\s*([0-9]+.[0-9]?)$/)
{
+ $val= $3;
+ $idx= $4;
+ ($bench = $1) =~ s/\s+$//;
+ $$results_ref-{$bench}{Result}{$par}= $val;
+ $$results_ref-{$bench}{Index}{$par}= $idx;
+}
+next;
+ }
+ close($h);
+}
+
+sub unixbench_print_results ($$) {
+ my ($results,$rfilen)= @_;
+ open my $h, |-, tee $rfilen or die $!;
+
+ printf $h %-50s,\BENCHMARK NAME\;
+ foreach my $i (sort keys $results-{'Double-Precision Whetstone'}{Index}) {
+printf $h %-15s,\$i VCPUs\;
+ }
+ print $h \n;
+ foreach my $b (keys $results) {
+printf $h %-50s,\$b\;
+foreach my $i (sort keys $results-{$b}{Index}) {
+ printf $h %-15s,$results-{$b}{Index}{$i};
+}
+print $h \n;
+ }
+ close($h);
+}
+
+sub unixbench_plot_results ($$$) {
+ my ($dataf,$num_cols,$pfile)= @_;
+ my $h= new IO::File $pfile.gp or die $!;
+
+ printf $h EOF;
+set terminal png enhanced font
/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf 8 size 800,600
+set output '$pfile.png'
+set title 'Unixbench INDEXes for $flight.$job'
+set key outside center top horizontal noreverse noenhanced autotitles nobox
+set xtics mirror rotate by -45 out
+set style data histogram
+set style histogram cluster gap 1
+set style fill solid border lt -1
+set boxwidth 1 absolute
+set bmargin 13
+set rmargin 14
+SKIP_COL=1
+NCOL=$num_cols
+HWIDTH=1.0/(NCOL+1.0)
+plot for [c=SKIP_COL+1:SKIP_COL+NCOL] '$dataf' using c:xtic(1) with
histograms title columnhead, \\
+for [c=SKIP_COL+1:SKIP_COL+NCOL]'' every ::1 using 0:c:c with labels
notitle offset first -HWIDTH*(NCOL/2.0)+HWIDTH/2.0+(c-(SKIP_COL+1))*HWIDTH,
character 2 rotate by 90
+EOF
+ close($h);
+
+ my $gp= can_run('gnuplot') or return;
Need to install gnuplot
Re: [Xen-devel] [PATCH 09/27] ts-unixbench-reslts: process and plot bench results
On Thu, 2014-12-11 at 12:15 +, Wei Liu wrote:
On Wed, Dec 10, 2014 at 07:10:06PM +0100, Dario Faggioli wrote:
This is done in a new Osstest/Benchmarking.pm module, as
the functions introduced may turn out useful somewhere else
too.
I would suggest using a dedicated commit for the introduction of
Benchmarking.pm.
I'm not sure. I like it being introduced here where it's useful, since
that does not, IMO, make the patch too long or to difficult to
understand/review/etc.
However, this is not a too big deal... let's see what others think, and
I'll copy. :-)
--- /dev/null
+++ b/Osstest/Benchmarking.pm
@@ -0,0 +1,115 @@
+# This is part of osstest, an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
2009-2014
Right! :-)
+sub unixbench_plot_results ($$$) {
+ my ($dataf,$num_cols,$pfile)= @_;
+ my $h= new IO::File $pfile.gp or die $!;
+
+ printf $h EOF;
+set terminal png enhanced font
/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf 8 size 800,600
+set output '$pfile.png'
+set title 'Unixbench INDEXes for $flight.$job'
+set key outside center top horizontal noreverse noenhanced autotitles nobox
+set xtics mirror rotate by -45 out
+set style data histogram
+set style histogram cluster gap 1
+set style fill solid border lt -1
+set boxwidth 1 absolute
+set bmargin 13
+set rmargin 14
+SKIP_COL=1
+NCOL=$num_cols
+HWIDTH=1.0/(NCOL+1.0)
+plot for [c=SKIP_COL+1:SKIP_COL+NCOL] '$dataf' using c:xtic(1) with
histograms title columnhead, \\
+for [c=SKIP_COL+1:SKIP_COL+NCOL]'' every ::1 using 0:c:c with
labels notitle offset first
-HWIDTH*(NCOL/2.0)+HWIDTH/2.0+(c-(SKIP_COL+1))*HWIDTH, character 2 rotate
by 90
+EOF
+ close($h);
+
+ my $gp= can_run('gnuplot') or return;
Need to install gnuplot before hand?
Well, this is the OSSTest controller, not the target host, guest or
anything, so I'd need to install it by some perl equivalent of
'system(apt-get install gnuplot) (or whatever Perl offers to run
system commands).
I haven't found many examples of executing commands on the controller in
current OSSTest codebase, and I'm not sure whether that is a good idea
and/or a strong requirement here... Again, let's see what others think.
+ my ($ok,$err)= run( command = $gp $pfile.gp, verbose = 1 );
+ logm(WARNING: plotting file with \$err\) unless $ok;
Fail the test case instead of issuing a warning?
You think? Well, the benchmark(s) did run, and produced results that we
can analyze in many way other than plotting, either inside OSSTest
(adding the logic for that, of course) or outside of it, so I wouldn't
call this a 'failure'.
Perhaps I should distinguish between the cases where gnuplot is just not
there, in which case I really would continue just issuing warnings, from
the case where the plotting command ran and failed, in which cased I
agree, we should fail the test... What do you think?
Thanks and Regards,
Dario
--
This happens because I choose it to happen! (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems RD Ltd., Cambridge (UK)
signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 09/27] ts-unixbench-reslts: process and plot bench results
On Thu, Dec 11, 2014 at 02:11:44PM +0100, Dario Faggioli wrote:
[...]
+sub unixbench_plot_results ($$$) {
+ my ($dataf,$num_cols,$pfile)= @_;
+ my $h= new IO::File $pfile.gp or die $!;
+
+ printf $h EOF;
+set terminal png enhanced font
/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf 8 size 800,600
+set output '$pfile.png'
+set title 'Unixbench INDEXes for $flight.$job'
+set key outside center top horizontal noreverse noenhanced autotitles
nobox
+set xtics mirror rotate by -45 out
+set style data histogram
+set style histogram cluster gap 1
+set style fill solid border lt -1
+set boxwidth 1 absolute
+set bmargin 13
+set rmargin 14
+SKIP_COL=1
+NCOL=$num_cols
+HWIDTH=1.0/(NCOL+1.0)
+plot for [c=SKIP_COL+1:SKIP_COL+NCOL] '$dataf' using c:xtic(1) with
histograms title columnhead, \\
+for [c=SKIP_COL+1:SKIP_COL+NCOL]'' every ::1 using 0:c:c with
labels notitle offset first
-HWIDTH*(NCOL/2.0)+HWIDTH/2.0+(c-(SKIP_COL+1))*HWIDTH, character 2 rotate
by 90
+EOF
+ close($h);
+
+ my $gp= can_run('gnuplot') or return;
Need to install gnuplot before hand?
Well, this is the OSSTest controller, not the target host, guest or
anything, so I'd need to install it by some perl equivalent of
'system(apt-get install gnuplot) (or whatever Perl offers to run
system commands).
Ah, the controller, OK...
I haven't found many examples of executing commands on the controller in
current OSSTest codebase, and I'm not sure whether that is a good idea
and/or a strong requirement here... Again, let's see what others think.
+ my ($ok,$err)= run( command = $gp $pfile.gp, verbose = 1 );
+ logm(WARNING: plotting file with \$err\) unless $ok;
Fail the test case instead of issuing a warning?
You think? Well, the benchmark(s) did run, and produced results that we
can analyze in many way other than plotting, either inside OSSTest
(adding the logic for that, of course) or outside of it, so I wouldn't
call this a 'failure'.
Since gnuplot runs on controller this point becomes moot.
Perhaps I should distinguish between the cases where gnuplot is just not
there, in which case I really would continue just issuing warnings, from
the case where the plotting command ran and failed, in which cased I
agree, we should fail the test... What do you think?
Make sense.
Wei.
Thanks and Regards,
Dario
--
This happens because I choose it to happen! (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems RD Ltd., Cambridge (UK)
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
