Greetings all,
I have been in the process of reviewing feedback and planning the project.
I believe I have come to a good purpose statement for OSCAR Bench:
A framework that allows for benchmarks and associated tools to be
installed, configured, compiled, and run using XML and Perl.
The framework is based off the idea that benchmarks typically read a
configuration file and then execute based off that. Also when
compiling, there is generally some user input required that is then
written somewhere.
I have broken the process down as follows:
install
compile
execute
results
1) Install
Install is getting the required files, and placing them where OSCAR
Bench can find them. OSCAR Package can handle this easily.
assuming OSCAR Bench is installed to /opt/oscar_bench
you would have:
++oscar_bench
++++benchmarks
++++docs
++++scripts
++++src
The 'benchmark' directory would be where you install benchmarks.
++oscar_bench
++++benchmarks
++++++bin
++++++doc
++++++scripts
++++++results
++++++src
++++++xml
The actual benchmark, for instance HPL, would be in
/opt/oscar_bench/benchmarks/hpl/src/hpl
or DARPA
/opt/oscar_bench/benchmarks/hpcc/src/hpcc
The folder xml will contain files that drive the compiling,
config/execution, and result processing.
under xml you would have:
compile.xml
config.xml
info.xml
results.xml
1) compile.xml will contain information pertaining to how to compile the
software. It may contain a series of questions the user needs to answer,
as well as provide a default value. You may also associate a script with
each question. The script allows the packager to return a set of dynamic
values that would be a good default choice. You may also specify which
line this information should be written to, or if line numbers are not
critical, then it can be skipped.
<?xml version="1.0" encoding="ISO-8859-1"?>
<compile>
<require_user_input>true</require_user_input>
#This specifies input values/text should be written to specific line
numbers
<use_line_numbers>true</use_line_numbers>
#Where to write this information
<filename>my_Makefile</filename>
<pre_script>do_something_before.pl</pre_script>
<post_script>do_something_before.pl</post_script>
<inputs>
<input>
#Hook that lets you provided dynamic configuration value
#For instance trying to find BLAS
<auto_config>benchmarks/hpl/scripts/find_BLAS.pl</auto_config>
#Specifies the line number this entry should go on
<line>0</line>
#Possibly the config file would need something like:
#X = value
#Here you could place:
#X =
#Whatever the use inputs would be placed following this text
<pre_value_text>
N =
</pre_value_text>
#This text would be entered following the value,
#if the value needs to be delimited by a semicolon
#or if you wish to entered a comment that could go here.
<post_value_text>
#A comment, or whatever may be required after a value
</post_value_text>
#The question the user will answer
<description>
How large should N be for HPL?
</description>
</input>
#another input....
<input>
<auto_config></auto_config>
<line></line>
<pre_value_text></pre_value_text>
<post_value_text></post_value_text>
<description></description>
<default></default>
</input>
</inputs>
</compile>
config.xml would be very similar. The idea being that you can define
what the software needs to know about compiling or running in XML. And
then the framework will ask the user the questions, and generate a file
in the end. The pre_script, post_script allow the packager to do tasks
before or after the asking the user input. In the end a file is written,
and using the post_script you can execute a command that would need the
file: make, mpirun, qsub....
++oscar_bench
++++benchmarks
++++++bin
++++++doc
++++++scripts
++++++results
++++++src
++++++xml
++++++++compile.xml
++++++++config.xml
++++++++info.xml
++++++++results.xml
The info.xml and results.xml do not involve user input, info.xml
contains information about the tool you are installing. This type of
info is what the user can see to know whether they want to install this
or not.
What does it do, and what is its license, author, website, etc...
results.xml defines how results are handled. It should include a
database schema so the results can be stored, and it should include
descriptive text for each result generated: What is MPI Latency?
I also want to include hooks so the packager can include dynamic 'good
values' that the results should match up to: if you are doing disk IO
possibly detect whether its ATA or SATA. This type of scripting is
optional and may or may not be of any use.
under oscar_bench,
docs should contain information on using OSCAR Bench
under each benchmark there is also docs, this should contain any
documentation for that specific benchmark, or possibly further reading
that is to involved to display in a Qt/command line enviroment.
each benchmark also has a bin directory, this is where the benchmark
will be compiled to ( if it needs to be compiled ), or where the
executable program/files should be.
I am 100% positive I will change and adapt this as I go, but my goal is
to provide a way to easily execute tools. If you want an NFS benchmark,
you would write any required scripts, and then define the XML files. If
no user input is needed, you can leave them out, if no dynamic values
are needed you may leave them out. Then create your benchmark to read
from a file if it needs values, and it will plug right in.
I appreciate any criticism, it took way to long to write this email, as
I continually feel I am leaving something out. What is the best way to
communicate? It is challenging when you can't talk face to face =)
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Oscar-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oscar-devel
