Folks,
I had this wacky idea the other day.
I have a collection of scripts that I've accumulated to run
MTT with parameter combinations that MTT does not iterate on
(see #245). E.g., I have a dozen or so scripts that do
variations on something like this:
Do a 32-bit mtt run with compiler X, on 2 nodes
Do a 32-bit mtt run with compiler Y, on 2 nodes
Do a 64-bit mtt run with compiler X, on 2 nodes
Do a 64-bit mtt run with compiler Y, on 2 nodes
Do a 32-bit mtt run with compiler X, on 1 node
Do a 32-bit mtt run with compiler Y, on 1 node
Do a 64-bit mtt run with compiler X, on 1 node
Do a 64-bit mtt run with compiler Y, on 1 node
...
So I thought, "can I somehow harness xargs to do this
instead of creating another shell script every time I have
another testing combination I want to run?" xargs doesn't
seem powerful enough. Then I realized that the MTT run
engine *itself* does exactly what I want. Xargs on steroids.
So I created the following "MPI" details section to get MTT
to run itself:
[MPI Details: test-matrix]
exec = <<EOT
client/mtt
--debug
--trial
--file @INI_NAME@
--section "mpi.get.*ompi-nightly-trunk mtt mpi.details.*open.mpi
reporter.*iu.database mpi.install.*autotools ..."
--scratch
/opt/mtt/@bitness@/@compiler@/&mpi_details_name()/my-scratch-area
prepend_path="PATH /path/to/my/compiler/@compiler@"
bitness=@bitness@
run_on_single_node=@run_on_single_node@
EOT
bitness = &enumerate(32,64)
run_on_single_node = &enumerate(0, 1)
compiler = &enumerate("A", "B")
The above expands to 8 MTT runs (2 compilers x 2 bitnesses x
2 hostlists). Then, I had to fake MTT out in order to *use*
the run engine. So I created some Noop.pm modules that do
nothing but scoot MTT along to the RunEngine phase:
[MPI get: noop]
# Fill this in from the command line
mpi_details =
module = Noop
[MPI install: noop]
mpi_get = noop
module = Noop
[Test get: noop]
module = Noop
[Test build: noop]
test_get = noop
module = Noop
[Test run: noop]
test_build = noop
module = Noop
timeout = -1
Finally, I can kick off a series of MTT runs like this:
$ client/mtt --section "noop test-matrix" mpi_details=test-matrix
What do you think?
I'd like to make this use case even easier. E.g., create a
--run-engine (?) option for the specific purpose of skipping
all the phases in order to simply execute commands. Or maybe
there is another tool to do what I am trying to do? In
which case, this has been an exercise :-)
-Ethan
