Re: [MTT users] Why is this loop_spawn test result marked FAIL?

[Ethan Mallove]

I've filed a Trac ticket for this issue:

http://svn.open-mpi.org/trac/mtt/ticket/393

On Wed, Apr/14/2010 11:36:04AM, Jeff Squyres wrote:
> That's interesting -- I see "Failed; exit status: 0" in the "Result message" 
> field.
> 
> Looks like that's coming from Test::Analyze::Correctness.  Somehow $pass came 
> out to be 0, so it fell into the "fail" path...?
> 
> 
> On Apr 14, 2010, at 10:51 AM, Ethan Mallove wrote:
> 
> > This loop_spawn test appears to have passed, though it's marked
> > as FAIL:
> > 
> >   http://www.open-mpi.org/mtt/index.php?do_redir=1233
> > 
> > Its pass condition is set to:
> > 
> >   pass = (_wifexited(), (_wexitstatus(), 0))
> > 
> > -Ethan
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> > 
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] [MTT bugs] [MTT] #212: Genericnetworklockingserver *REVIEW NEEDED*

[Ethan Mallove]

I can do the below stuff in a seperate commit.

-Ethan

On Fri, Mar/05/2010 09:26:21PM, Jeff Squyres wrote:
> On Mar 5, 2010, at 2:05 PM, Ethan Mallove wrote:
> 
> > The check is there now.  Ready for review.
> 
> Did you mean to include these parts?
> 
> --- lib/MTT/Messages.pm Mon Nov 09 14:38:09 2009 -0500
> +++ lib/MTT/Messages.pm Fri Mar 05 14:02:39 2010 -0500
> @@ -174,7 +174,7 @@
>  $lev = 0 if (! defined($lev));
>  my @called = caller($lev);
>  
> -my $s = wrap("", "   ", (join(":", map { &_relative_path($_) } 
> @called[1..2]), @_));
> +my $s = (join(":", map { &_relative_path($_) } @called[1..2]), @_);
>  print $s;
>  print $LOGFILE $s
>  if (defined($LOGFILE));
> 
> --- client/mtt  Mon Nov 09 14:38:09 2009 -0500
> +++ client/mtt  Fri Mar 05 14:02:39 2010 -0500
> @@ -498,6 +498,15 @@
>  # execute on_start callback if exists
> _do_step($ini, "mtt", "before_mtt_start_exec");
>  
> +# Process setenv, unsetenv, prepend_path, and append_path
> +
> +my $config;
> +$config->{setenv} = Value($ini, "mtt", "setenv");
> +$config->{unsetenv} = Value($ini, "mtt", "unsetenv");
> +$config->{prepend_path} = Value($ini, "mtt", "prepend_path");
> +$config->{append_path} = Value($ini, "mtt", "append_path");
> +my @save_env;
> +ProcessEnvKeys($config, \@save_env);
>  
>  # Set the logfile, if specified
>  
> --- lib/MTT/Defaults.pm Mon Nov 09 14:38:09 2009 -0500
> +++ lib/MTT/Defaults.pm Fri Mar 05 14:02:39 2010 -0500
> @@ -42,7 +42,7 @@
>  
>  known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai", "absoft",
>"pathscale", "sun", "microsoft", "none", 
> "unknown" ],
> -known_resource_manager_names => [ "slurm", "tm", "loadleveler", "n1ge",
> +known_resource_manager_names => [ "slurm", "tm", "loadleveler", "sge",
>"alps", "none", "unknown" ],
>  known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx", "verbs",
>   "udapl", "psm", "elan", "portals", "shmem",
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] [MTT bugs] [MTT] #212: Generic networklockingserver *REVIEW NEEDED*

[Ethan Mallove]

On Fri, Feb/19/2010 12:00:55PM, Ethan Mallove wrote:
> On Thu, Feb/18/2010 04:13:15PM, Jeff Squyres wrote:
> > On Feb 18, 2010, at 10:48 AM, Ethan Mallove wrote:
> > 
> > > To ensure there is never a collision between $a->{k} and $b->{k}, the
> > > user can have two MTT clients share a $scratch, but they cannot both
> > > run the same INI section simultaneously.  I setup my scheduler to run
> > > batches of MPI get, MPI install, Test get, Test build, and Test run
> > > sections in parallel with successor INI sections dependent on their
> > > predecessor INI sections (e.g., [Test run: foo] only runs after [Test
> > > build: foo] completes).  The limitation stinks, but the current
> > > limitation is much worse: two MTT clients can't even run the same
> > > *phase* out of one $scratch.
> > 
> > Maybe it might be a little nicer just to protect the user from
> > themselves -- if we ever detect a case where $a->{k} and $b->{k}
> > both exist and are not the same value, dump out everything to a file
> > and abort with an error message.  This is clearly an erroneous
> > situation, but running MTT in big parallel batches like this is a
> > worthwhile-but-complicated endeavor, and some people are likely to
> > get it wrong.  So we should at least detect the situation and fail
> > gracefully, rather than losing or corrupting results.
> > 
> > Make sense?
> 
> Yes.  I'll add this.

The check is there now.  Ready for review.

-Ethan

> 
> -Ethan
> 
> > 
> > > I originally wanted the .dump files to be completely safe, but MTT
> > > clients were getting locked out of the .dump files for way too long.
> > > E.g., MTT::MPI::LoadInstalls happens very early in client/mtt, and an
> > > hour could elapse before MTT::MPI::SaveInstalls is called in
> > > Install.pm.
> > 
> > Yep, if you lock from load->save, then that can definitely happen...
> > 
> > -- 
> > Jeff Squyres
> > jsquy...@cisco.com
> > 
> > For corporate legal information go to:
> > http://www.cisco.com/web/about/doing_business/legal/cri/
> > 
> > 
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
--- client/mtt  Mon Nov 09 14:38:09 2009 -0500
+++ client/mtt  Fri Mar 05 14:02:39 2010 -0500
@@ -498,6 +498,15 @@
 # execute on_start callback if exists
_do_step($ini, "mtt", "before_mtt_start_exec");

+# Process setenv, unsetenv, prepend_path, and append_path
+
+my $config;
+$config->{setenv} = Value($ini, "mtt", "setenv");
+$config->{unsetenv} = Value($ini, "mtt", "unsetenv");
+$config->{prepend_path} = Value($ini, "mtt", "prepend_path");
+$config->{append_path} = Value($ini, "mtt", "append_path");
+my @save_env;
+ProcessEnvKeys($config, \@save_env);

 # Set the logfile, if specified

--- lib/MTT/Defaults.pm Mon Nov 09 14:38:09 2009 -0500
+++ lib/MTT/Defaults.pm Fri Mar 05 14:02:39 2010 -0500
@@ -42,7 +42,7 @@

 known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai", "absoft",
   "pathscale", "sun", "microsoft", "none", 
"unknown" ],
-known_resource_manager_names => [ "slurm", "tm", "loadleveler", "n1ge",
+known_resource_manager_names => [ "slurm", "tm", "loadleveler", "sge",
   "alps", "none", "unknown" ],
 known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx", "verbs",
  "udapl", "psm", "elan", "portals", "shmem",
--- lib/MTT/MPI.pm  Mon Nov 09 14:38:09 2009 -0500
+++ lib/MTT/MPI.pm  Fri Mar 05 14:02:39 2010 -0500
@@ -16,6 +16,8 @@

 use strict;
 use MTT::Files;
+use MTT::Messages;
+use MTT::Util;

 #--

@@ -28,10 +30,13 @@
 #--

 # Filename where list of MPI sources is kept
-my $sources_data_filename = "mpi_sources.dump";
+my $sources_data_filename = "mpi_sources";

 # File

Re: [MTT users] [MTT bugs] [MTT] #212: Generic network locking server *REVIEW NEEDED*

[Ethan Mallove]

This apparently got lost in the shuffle a few months ago.  The fix
allows one to kick off all of their MPI Installs and Test Builds in
parallel.  Give it a try when you have a chance.

-Ethan


> On Sat, Nov/07/2009 04:15:42PM, Jeff Squyres wrote:
> > On Nov 6, 2009, at 5:18 PM, Ethan Mallove wrote:
> >
> >> I'm running multiple MTT clients out of the same scratch directory
> >> using SGE. I'm running into race conditions between the multiple
> >> clients, where one client is overwriting another's data in the .dump
> >> files - which is a Very Bad Thing(tm). I'm running the
> >> client/mtt-lock-server, and I've added the corresponding [Lock]
> >> section in my INI file. Will my MTT clients now not interfere with
> >> each other's .dump files? I'm skeptical of this because I don't see,
> >> e.g., Lock() calls in SaveRuns(). How do I make my .dump files safe?
> >>
> >
> >
> > Err... perhaps this part wasn't tested well...?
> >
> > I'm afraid it's been forever since I've looked at this code and I'm gearing 
> > up to leave for the Forum on Tuesday and then staying on for SC09, so it's 
> > quite likely that you'll be able to look at this in more detail before I 
> > will.  Sorry to pass the buck; just trying to be realistic...  :-(
> 
> After some digging, I discover that MTT is not designed to execute
> multiple INI sections out of a single scratch directory in parallel.
> There's a ticket for this:
> 
>   https://svn.open-mpi.org/trac/mtt/ticket/167
> 
> The way around this limitation is to have MTT split up the .dump files
> by INI section so that two MTT client running simultaneously never
> conflict with each other. (This change did not need to be made for the
> Test run .dump files, as MTT already splits them up.) I have attached
> a patch, which makes a simple wrapper script for #167 possible. The
> changes should not disrupt normal (non-parallel) execution. Anyone
> care to give it a try?
> 
> -Ethan
> 
> >
> > -- 
> > Jeff Squyres
> > jsquy...@cisco.com
> >
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
diff -r 8760c5d19838 -r 8a8663cb0ac3 lib/MTT/MPI.pm
--- a/lib/MTT/MPI.pmMon Nov 09 14:38:09 2009 -0500
+++ b/lib/MTT/MPI.pmWed Nov 18 11:07:37 2009 -0500
@@ -16,6 +16,8 @@

 use strict;
 use MTT::Files;
+use MTT::Messages;
+use MTT::Util;

 #--

@@ -28,10 +30,13 @@
 #--

 # Filename where list of MPI sources is kept
-my $sources_data_filename = "mpi_sources.dump";
+my $sources_data_filename = "mpi_sources";

 # Filename where list of MPI installs is kept
-my $installs_data_filename = "mpi_installs.dump";
+my $installs_data_filename = "mpi_installs";
+
+# Filename extension for all the Dumper data files
+my $data_filename_extension = "dump";

 #--

@@ -42,10 +47,15 @@
 # Explicitly delete anything that was there
 $MTT::MPI::sources = undef;

-# If the file exists, read it in
-my $data;
-MTT::Files::load_dumpfile("$dir/$sources_data_filename", \$data);
-$MTT::MPI::sources = $data->{VAR1};
+my @dumpfiles = 
glob("$dir/$sources_data_filename-*.$data_filename_extension");
+foreach my $dumpfile (@dumpfiles) {
+
+# If the file exists, read it in
+my $data;
+MTT::Files::load_dumpfile($dumpfile, \$data);
+$MTT::MPI::sources = MTT::Util::merge_hashes($MTT::MPI::sources, 
$data->{VAR1});
+
+}

 # Rebuild the refcounts
 foreach my $get_key (keys(%{$MTT::MPI::sources})) {
@@ -62,9 +72,14 @@
 #--

 sub SaveSources {
-my ($dir) = @_;
+my ($dir, $name) = @_;

-MTT::Files::save_dumpfile("$dir/$sources_data_filename", 
+# We write the entire MPI::sources hash to file, even
+# though the filename indicates a single INI section
+# MTT::Util::hashes_merge will take care of duplicate
+# hash keys. The reason for splitting up the .dump files
+# is to keep them read and write safe across INI sections
+
MTT::Files::save_dumpfile("$dir/$sources_data_filename-$name.$data_filename_extension",
 
   $MTT::MPI::sources);
 }

@@ -76,10 +91,14 @@
 # Explicitly delete anything that was there
 $MTT::MPI::installs = undef;

-# If the file exists, read it in
-my $data;
-MTT::Files::load_dumpfile("$dir/$installs_data_filename&quo

Re: [OMPI users] ompi_info loop

[Ethan Mallove]

On Tue, Feb/09/2010 08:46:53AM, Benjamin Gaudio wrote:
> In trying to track down my default hostfile problem, I found that
> when I run ompi_info, it simply keeps repeating:
> 
> Displaying Open MPI information for 32-bit ...
> Displaying Open MPI information for 32-bit ...
> Displaying Open MPI information for 32-bit ...
> Displaying Open MPI information for 32-bit ...
> Displaying Open MPI information for 32-bit ...
> Displaying Open MPI information for 32-bit ...
> 
> Has anyone seen this before?  I am using Sun HPC ClusterTools 8.2.1
> on Solaris 10.

Hi Benjamin,

I can reproduce this using /opt/SUNWhpc/HPC8.2.1/sun/bin/64/ompi_info.
We'll fix this ASAP (for the next ClusterTools release).  A workaround
is to use /opt/SUNWhpc/HPC8.2.1/sun/bin/32/ompi_info.

Thanks,
Ethan

> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [MTT users] MTT:Error while executing ompi-core-perf-testing.ini

[Ethan Mallove]

Hi Vishal,

The debug output shows that your MTT run aborted because you do not
have a username/passwd for the open-mpi.org/mtt/submit URL. To skip
the IU Database part, try running with --no-section iu.database, e.g.,

  # cat ompi-core-perf-testing.ini | ../client/mtt --no-section iu.database 
--debug -

Thanks,
Ethan

On Mon, Jan/04/2010 05:08:36PM, vishal shorghar wrote:
> Hi All,
>
> Today I ran ompi-core-perf-test script as "# cat  
> ompi-core-perf-testing.ini | ../client/mtt  --debug -"
> It gave me following error and exits .
>
> [root@veera samples]# cat  ompi-core-perf-testing.ini | ../client/mtt  
> --debug -
> Debug is 1, Verbose is 1
>*** MTT: ../client/mtt --debug - 
>   *** Running on veera  
>  Chdir ../client
> Chdir /root/mtt-svn/samples 
>Copying: stdin to /tmp/dw1zvwGZEa.ini
>   Expanding include_file(s) parameters in 
> /tmp/dw1zvwGZEa.iniReading ini file: stdin  
>   Validating INI inifile: 
> /tmp/bH9q7NY7Hl.iniFilterINI: Final 
> list of sections:   [mtt]   
>   
> [lock]  
>   [mpi get: ompi-nightly-trunk] 
> [mpi get: ompi-nightly-v1.2]
>   [mpi get: ompi-released-v1.2] 
> [mpi get: mpich1]   
>   [mpi get: mpich-mx]   
> [mpi get: mpich2]   
>   [mpi get: mvapich1]   
> [mpi get: mvapich2] 
>   [mpi get: hp mpi] 
> [mpi get: intel mpi]
>   [skip mpi get: scali mpi] 
> [skip mpi get: cray mpi]
>   [mpi install: ompi/gnu-standard]  
> [mpi install: mpich1]   
>   [mpi install: mpich2] 
> [mpi install: mvapich1] 
>   [mpi install: mvapich2]   
> [mpi install: intel mpi]
>   [mpi install: hp mpi] 
> [mpi details: ompi] 
>   [mpi details: mpich1] 
> [mpi details: mpich2]   
>   [mpi details: mvapich1]   
> [mpi details: mvapich2] 
>   [mpi details: mpich-mx]   
> [mpi details: intel 
> mpi]  [mpi details: hp 
> mpi] [test get: 
> netpipe]   [test 
> get: osu]   
> [test get: imb] 
>   [test get: skampi]
> [test get: nbcbench]
>   [test build: netpipe] 
> [test build: osu]   
>   [test build: imb] 
> [test build: skampi]
>   [test build: nbcbench]
> [test run: netpipe] 
>   [test run: osu]   
> [test run: imb] 
>   [test run: skampi]
> [test run: nbcbench]

Re: [OMPI users] MTT -trivial :All tests are not getting passed

[Ethan Mallove]

Hi Vishal,

This is an MTT question for mtt-us...@open-mpi.org (see comments
below).

On Tue, Dec/22/2009 03:54:08PM, vishal shorghar wrote:
>Hi All,
> 
>I have one issue with MTT trivial tests.All tests are not getting
>passed,Please read below for detailed description.
> 
>Today I ran mtt trivial tests with latest ofed package
>OFED-1.5-20091217-0600 (ompi-1.4), between two machines,I was  able to run
>the MTT trivial tests manually but not through MTT framework. I think we
>are missing some configuration steps since it is unable to find the test
>executables in the test run phase of the MTT.
> 
>-> When we ran it through MTT it gave us the error and exits.
>I ran the test as  "cat developer.ini trivial.ini | ../client/mtt
>--verbose - "
> 
>-> When we analyzed error from
>/root/mtt-svn/samples/Test_Run-trivial-my_installation-1.4.txt file we
>found it is not getting the executable files of the different test to
>execute.
> 
>-> Then we found that those executables were being generated only on one
>of the machine out of two machines. So, we manually copied the tests  from
>/root/mtt-svn/samples/installs/nRpF/tests/trivial/test_get__trivial/c_ring
>to another machine.
> 
>-> And we ran it manually as shown below and it worked fine:
>mpirun --host 102.77.77.64,102.77.77.68 -np 2 --mca btl openib,sm,self
>--prefix
>
> /usr/mpi/gcc/openmpi-1.4/root/mtt-svn/samples/installs/nRpF/tests/trivial/test_get__trivial/c_ring
> 
>-> I am attaching file trivial.ini,developer.ini and
>/root/mtt-svn/samples/Test_Run-trivial-my_installation-1.4.txt.
> 
>Let us know if I am  missing some configuration steps.
> 

You need to set your scratch directory (via the --scratch option) to
an NFS share that is accessible to all nodes in your hostlist.  MTT
won't copy local files onto each node for you.

Regards,
Ethan


>NOTE:
>
>It gave me following output at the end of execution of test command and
>the same is saved in /root/mtt-svn/samples/All_phase-summary.txt
> 
>hostname: nizam
>uname: Linux nizam 2.6.18-128.el5 #1 SMP Wed Jan 21 10:41:14 EST 2009
>x86_64 x86_64 x86_64 GNU/Linux
>who am i:
> 
>
> +-+-+-+--+--+--+--+--+--+
>| Phase   | Section | MPI Version | Duration | Pass | Fail |
>Time out | Skip | Detailed report  |
>
> +-+-+-+--+--+--+--+--+--+
>| MPI Install | my installation | 1.4 | 00:00| 1|
>|  |  | MPI_Install-my_installation-my_installation-1.4.html |
>| Test Build  | trivial | 1.4 | 00:01| 1|
>|  |  | Test_Build-trivial-my_installation-1.4.html  |
>| Test Run| trivial | 1.4 | 00:10|  | 8
>|  |  | Test_Run-trivial-my_installation-1.4.html|
>
> +-+-+-+--+--+--+--+--+--+
> 
>Total Tests:10
>Total Failures: 8
>Total Passed:   2
>Total Duration: 11 secs. (00:11)
> 
>Thanks  & Regards,
> 
>Vishal shorghar
>MTS
>Chelsio Communication

> #
> # Copyright (c) 2007 Sun Microystems, Inc.  All rights reserved.
> #
> 
> # Template MTT configuration file for Open MPI developers.  The intent
> # for this template file is to establish at least some loose
> # guidelines for what Open MPI core developers should be running
> # before committing changes to the OMPI repository. This file is not
> # intended to be an exhaustive sample of all possible fields and
> # values that MTT offers. Each developer will undoubtedly have to
> # edit this template for their own needs (e.g., pick compilers to use,
> # etc.), but this file provides a baseline set of configurations that
> # we intend for you to run.
> #
> # Sample usage:
> #   cat developer.ini intel.ini   | client/mtt - 
> alreadyinstalled_dir=/your/install
> #   cat developer.ini trivial.ini | client/mtt - 
> alreadyinstalled_dir=/your/install
> #
> 
> [MTT]
> # No overrides to defaults
> 
> # Fill this field in
> 
> #hostlist = 102.77.77.63 102.77.77.54 102.77.77.64 102.77.77.68 
> #hostlist = 102.77.77.66 102.77.77.68 102.77.77.63 102.77.77.64 102.77.77.53 
> 102.77.77.54 102.77.77.243 102.77.77.65
> hostlist = 102.77.77.64 102.77.77.68 
> hostlist_max_np = 2 
> max_np = 2
> force = 1
> #prefix = /usr/mpi/gcc/openmpi-1.3.4/bin
> 
> #--
> 
> [MPI Details: Open MPI]
> 
> exec = mpirun @hosts@ -np _np() @mca@ --prefix _prefix() 
> _executable() _argv()
> 
> mca = 

Re: [MTT users] MTT trivial tests failstocomplete

[Ethan Mallove]

On Thu, Nov/19/2009 02:05:06PM, Venkat Venkatsubra wrote:
> Hi Ethan,
> 
> Thanks for your help.
> I now started with a clean developer.ini and made only the following
> change and reran.
> # Fill this field in
> hostlist = 102.77.77.63
> hostlist_max_np = 1
> max_np = 1
> 
> It doesn't hang like it used to before. I get the below error.
> ...
> ...
> Value got: Config::IniFiles=HASH(0x1ef499a0) mtt submit_group_results
> Value returning:
> Returining reference to an array of a single scalar
> *** ERROR: Could not evaluate: $ret =
> MTT::Values::Functions::if(MTT::Values::Functions::have_hostfile(),
> MTT::Values::Functions::join("--hostfile ",
> MTT::Values::Functions::hostfile()),
> MTT::Values::Functions::if(
> MTT::Values::Functions::have_hostlist(),
> MTT::Values::Functions::join("--host ",
> MTT::Values::Functions::hostlist()),
> ""
> )
> );: Undefined subroutine ::Values::Functions::join called at
> (eval 65) line 1,  line 3.
> 
> What should be the next developer.ini change that I should be making ?
> 
> The complete output and the developer.ini attached.

Oops. () should be ().

"svn up" and try again.

-Ethan

> 
> Thanks!
> 
> Venkat
> 
> -Original Message-
> From: mtt-users-boun...@open-mpi.org
> [mailto:mtt-users-boun...@open-mpi.org] On Behalf Of Ethan Mallove
> Sent: Thursday, November 19, 2009 11:32 AM
> To: General user list for the MPI Testing Tool
> Cc: Divya Manjunath
> Subject: Re: [MTT users] MTT trivial tests failstocompleteon
> Centos5.3-x86_64 bit platform with OFED 1.5
> 
> On Thu, Nov/19/2009 08:53:12AM, Venkat Venkatsubra wrote:
> > Hi Ethan,
> > 
> > Yes, we are using http://svn.open-mpi.org/svn/mtt/trunk.
> > After "svn update" when I rerun this is what the screen output looks
> > like.
> > 
> > Debug is 1, Verbose is 1
> > *** MTT: ../client/mtt --debug -
> > *** Running on mughal
> > Chdir ../client
> > Chdir /root/mtt-svn/samples
> > Copying: stdin to /tmp/XzxTdnyg85.ini
> > Expanding include_file(s) parameters in /tmp/XzxTdnyg85.ini
> > Reading ini file: stdin
> > *** ERROR: Syntax error in stdin: Line 36 in file /tmp/UqPeD61ett.ini
> is
> > mal-formed:
> > <<<<<<< .mine
> > Line 41 in file /tmp/UqPeD61ett.ini is mal-formed:
> > ===
> > Line 53 in file /tmp/UqPeD61ett.ini is mal-formed:
> > >>>>>>> .r1337 at /root/mtt-svn/lib/MTT/Messages.pm line
> > 121.
> 
> You have some SVN conflicts in your samples/developer.ini file.
> Do you have your own local changes to that file, e.g., near the
> "hosts" parameter? I think you just want to resolve those with:
> 
>   $ svn resolve --accept=theirs-conflict samples/developer.ini
> 
> -Ethan
> 
> > 
> > Attached are the new developer.ini and the /tmp/UqPeD61ett.ini files.
> > 
> > Venkat
> > 
> > -Original Message-
> > From: mtt-users-boun...@open-mpi.org
> > [mailto:mtt-users-boun...@open-mpi.org] On Behalf Of Ethan Mallove
> > Sent: Thursday, November 19, 2009 8:36 AM
> > To: General user list for the MPI Testing Tool
> > Subject: Re: [MTT users] MTT trivial tests fails tocompleteon
> > Centos5.3-x86_64 bit platform with OFED 1.5
> > 
> > On Wed, Nov/18/2009 12:17:24PM, Venkat Venkatsubra wrote:
> > > Attached.
> > 
> > That developer.ini file is out-of-date. Can you "svn update", and try
> > to run again? I assume the SVN URL you're using is
> > http://svn.open-mpi.org/svn/mtt/trunk?
> > 
> > -Ethan
> > 
> > > 
> > > -Original Message-
> > > From: mtt-users-boun...@open-mpi.org
> > > [mailto:mtt-users-boun...@open-mpi.org] On Behalf Of Ethan Mallove
> > > Sent: Wednesday, November 18, 2009 1:41 PM
> > > To: General user list for the MPI Testing Tool
> > > Subject: Re: [MTT users] MTT trivial tests fails to completeon
> > > Centos5.3-x86_64 bit platform with OFED 1.5
> > > 
> > > Could you run with --debug (instead of --verbose), and send the
> > > output.
> > > 
> > > Thanks,
> > > Ethan
> > > 
> > > On Wed, Nov/18/2009 11:08:18AM, Venkat Venkatsubra wrote:
> > > > 
> > > > 
> > > > 
> > > > 
> > > >From: Venkat Venkatsubra
> > > >Sent: Wednesday, November 18, 2009 12

Re: [MTT users] MTT trivial tests fails to complete on Centos5.3-x86_64 bit platform with OFED 1.5

[Ethan Mallove]

Could you run with --debug (instead of --verbose), and send the
output.

Thanks,
Ethan

On Wed, Nov/18/2009 11:08:18AM, Venkat Venkatsubra wrote:
> 
> 
> 
> 
>From: Venkat Venkatsubra
>Sent: Wednesday, November 18, 2009 12:54 PM
>To: 'mtt-us...@open-mpi.org'
>Subject: MTT trivial tests fails to complete on Centos5.3-x86_64 bit
>platform with OFED 1.5
> 
> 
> 
>Hello All,
> 
> 
> 
>How do I debug this problem ? Attached are the developer.ini and
>trivial.ini files.
> 
>I can provide any other information that you need.
> 
> 
> 
>[root@samples]# cat /etc/issue
> 
>CentOS release 5.3 (Final)
> 
>Kernel \r on an \m
> 
> 
> 
>[root@samples]# uname -a
> 
>Linux 2.6.18-128.el5 #1 SMP Wed Jan 21 10:41:14 EST 2009 x86_64 x86_64
>x86_64 GNU/Linux
> 
> 
> 
>I am running OFED-1.5-20091029-0617 daily build.
> 
> 
> 
>Started trivial tests using the following command:
> 
> 
> 
>[root@samples]# cat developer.ini trivial.ini | ../client/mtt --verbose -
> 
>
> 
>
> 
> >> Initializing reporter module: TextFile
> 
> *** Reporter initialized
> 
> *** MPI Get phase starting
> 
> >> MPI Get: [mpi get: my installation]
> 
>Checking for new MPI sources...
> 
>Using MPI in: /usr/mpi/gcc/openmpi-1.3.2/
> 
> *** WARNING: alreadyinstalled_mpi_type was not specified, defaulting to
> 
> "OMPI".
> 
>Got new MPI sources: version 1.3.2
> 
> *** MPI Get phase complete
> 
> *** MPI Install phase starting
> 
> >> MPI Install [mpi install: my installation]
> 
>Installing MPI: [my installation] / [1.3.2] / [my installation]...
> 
> >> Reported to text file
> 
> 
>/root/mtt-svn/samples/MPI_Install-my_installation-my_installation-1.3.2.htm
> 
>   l
> 
> >> Reported to text file
> 
> 
>/root/mtt-svn/samples/MPI_Install-my_installation-my_installation-1.3.2.txt
> 
>Completed MPI Install successfully
> 
> *** MPI Install phase complete
> 
> *** Test Get phase starting
> 
> >> Test Get: [test get: trivial]
> 
>Checking for new test sources...
> 
>Got new test sources
> 
> *** Test Get phase complete
> 
> *** Test Build phase starting
> 
> >> Test Build [test build: trivial]
> 
>Building for [my installation] / [1.3.2] / [my installation] /
>[trivial]
> 
> >> Reported to text file
> 
>   /root/mtt-svn/samples/Test_Build-trivial-my_installation-1.3.2.html
> 
> >> Reported to text file
> 
>   /root/mtt-svn/samples/Test_Build-trivial-my_installation-1.3.2.txt
> 
>Completed test build successfully
> 
> *** Test Build phase complete
> 
> *** Test Run phase starting
> 
> >> Test Run [trivial]
> 
> >> Running with [my installation] / [1.3.2] / [my installation]
> 
>Using MPI Details [open mpi] with MPI Install [my installation]
> 
> 
> 
>During this stage the test stalls.
> 
>After about ~10 minutes the test gets killed.
> 
>dmesg on which the test is running displays the following output:
> 
> 
> 
> ==
> 
> Dmesg output
> 
> ==
> 
> Out of memory: Killed process 5346 (gdmgreeter).
> 
> audispd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
> 
> 
> 
> Call Trace:
> 
>  [] out_of_memory+0x8e/0x2f5
> 
>  [] __alloc_pages+0x245/0x2ce
> 
>  [] __do_page_cache_readahead+0x95/0x1d9
> 
>  [] sock_readv+0xb7/0xd1
> 
>  [] __wake_up_common+0x3e/0x68
> 
>  [] filemap_nopage+0x148/0x322
> 
>  [] __handle_mm_fault+0x1f8/0xe5c
> 
>  [] do_page_fault+0x4cb/0x830
> 
>  [] error_exit+0x0/0x84
> 
> 
> 
>Thanks!
> 
> 
> 
>Venkat



> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] autogen.sh in svn_post_export

[Ethan Mallove]

On Wed, Nov/04/2009 02:22:28PM, Jeff Squyres wrote:
> On Nov 4, 2009, at 1:51 PM, Ethan Mallove wrote:
>
>> I recently ran into the error "./configure: No such file or directory"
>> in my nightly test build for the ibm, mpicxx, and onesided test
>> suites:
>>
>>   http://www.open-mpi.org/mtt/index.php?do_redir=1169
>>
>> This problem had something to do with autogen.sh being run in the Test
>> get section (different environment settings, maybe?). Moving the
>> autogen.sh into the Test build section (before "configure" in the
>> shell_build_command) resolved the problem. To prevent this problem,
>> would it be too uncouth to add pre-generated configure scripts (a la
>> the nightly Open MPI tarballs) in the ompi-tests repository?
>>
>
>
> Yes, it would be uncouth.  :-)  One does not put generated code in the 
> repository.
>
> I'm not sure why moving the autogen.sh from test get to test build would 
> fix the issue for you -- are your autotools in your normal environment, or 
> do you have them in some particular environment that is loaded by an mpi 
> install phase?

Hmmm. QLogic and IU are also running into the same issue:

  http://www.open-mpi.org/mtt/index.php?do_redir=1170

I just filed a Trac ticket:

  https://svn.open-mpi.org/trac/mtt/ticket/388

-Ethan

>
> -- 
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] autogen.sh in svn_post_export

[Ethan Mallove]

I recently ran into the error "./configure: No such file or directory"
in my nightly test build for the ibm, mpicxx, and onesided test
suites:

  http://www.open-mpi.org/mtt/index.php?do_redir=1169

This problem had something to do with autogen.sh being run in the Test
get section (different environment settings, maybe?). Moving the
autogen.sh into the Test build section (before "configure" in the
shell_build_command) resolved the problem. To prevent this problem,
would it be too uncouth to add pre-generated configure scripts (a la
the nightly Open MPI tarballs) in the ompi-tests repository?

-Ethan

Re: [MTT users] MTT Error on SLES11

[Ethan Mallove]

On Mon, Apr/13/2009 09:38:27AM, Rafael Folco wrote:
> On Thu, 2009-04-09 at 16:05 -0400, Ethan Mallove wrote:
> > On Wed, Apr/08/2009 05:05:30PM, Rafael Folco wrote:
> > > Ethan, Jeff,
> > > 
> > > Sorry for the perl confusion, nevermind...
> > > 
> > > Below is attached the log and ini files, and also how I run MTT.
> > > 
> > > Thanks,
> > > 
> > > Rafael
> > > 
> > > 
> > > client/mtt -d -v -p -f openmpi.ini 2>&1 |tee test.log
> > > 
> > > ...
> > > 
> > > 
> > > *** Run test phase starting
> > > >> Test run [trivial]
> > > Evaluating: trivial
> > > Found a match! trivial [trivial
> > > Evaluating: Simple
> > > >> Running with [openmpi-1.2.8] / [1.2.8] / [openmpi-1.2.8]
> > > Found MPI details: [mpi details: open mpi]
> > > Using [mpi details: open mpi] with [MPI Install: openmpi-1.2.8]
> > > Evaluating: # We can exit if the test passed or was skipped (i.e.,
> > > there's
> > >no need
> > ># to cleanup).
> > >if test "$MTT_TEST_RUN_RESULT" = "passed" -o "$MTT_TEST_RUN_RESULT" =
> > >"skipped"; then
> > >exit 0
> > >fi
> > >
> > >if test "$MTT_TEST_HOSTFILE" != ""; then
> > >args="--hostfile $MTT_TEST_HOSTFILE"
> > >elif test "$MTT_TEST_HOSTLIST" != ""; then
> > >args="--host $MTT_TEST_HOSTLIST"
> > >fi
> > >orterun $args -np $MTT_TEST_NP --prefix $MTT_TEST_PREFIX
> > >mtt_ompi_cleanup.pl
> > > Got final exec: mpirun (_hostfile(), "("--hostfile ",
> > >"()")",   "(_hostlist(), "("--host ",
> > >"()")", "")") -np _np() --mca btl openib,self --debug
> > >--prefix _prefix() _executable() _argv()
> > > chdir /tmp/ompi-core-testers/installs/dLS2/tests/trivial/test_get__trivial
> > > Evaluating: require MTT::Test::Specify::Simple
> > > Evaluating: $ret = ::Test::Specify::Simple::Specify(@args)
> > > Evaluating: _executables(".")
> > > Got name: find_executables
> > > Got args: "."
> > > _do: $ret = MTT::Values::Functions::find_executables(".")
> > > _executables got .
> > > _exectuables returning: ./c_ring ./f77_ring ./f90_hello ./cxx_ring
> > >./f77_hello ./cxx_hello ./f90_ring ./c_hello
> > > *** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
> > > string ("8") as an ARRAY ref while "strict refs" in use at
> > > /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> > 
> > You see this in both the "trunk" and "ompi-core-testers"? I think this 
> > problem with array references was ironed out in the trunk.
> 
> No. I see this error on ompi-core only. It looks like my INI file is not
> compliant with trunk. I've already tried on trunk and the Run phase is
> ignored. I'll take a look at the documentation to see what I am doing
> wrong.

I think this is your problem:

  ...
  *** WARNING: Unable to find MPI details section for [MPI Install: 
openmpi-1.2.8; skipping
  ...

Set your "mpi_details" param like this:

  mpi_details = Open MPI

The "hosts" needs to be changed as well. I have something very simple,
e.g.,

  hosts = --host host1 host2 host3

-Ethan


> 
> Thanks. 
> 
> > 
> > -Ethan
> > > 
> > > On Wed, 2009-04-08 at 14:15 -0400, Ethan Mallove wrote:
> > > > On Wed, Apr/08/2009 11:36:05AM, Rafael Folco wrote:
> > > > > Well, I took a look at /tmp/ompi-core-testers/lib/MTT/Values.pm line 
> > > > > 75.
> > > > > 
> > > > > This piece of code looks wrong to me:
> > > > > 
> > > > >  if ($#{@$ret} < 0) {
> > > > > 
> > > > > $ret references an array
> > > > > @$ret points to the first element of this array
> > > > > $# returns the number of elements
> > > > > 
> > > > > So this line is trying to count elements of the first element??! 
> > > > > Doesn't
> > > > > make sense. Correct me if I am wrong, what am I missing here ?
> > > > > 
> > > > > "if ($#{$ret} < 0) {" would be correct, without @. 
> > > > > 
> > > > > I bel

Re: [MTT users] MTT Error on SLES11

[Ethan Mallove]

On Wed, Apr/08/2009 05:05:30PM, Rafael Folco wrote:
> Ethan, Jeff,
> 
> Sorry for the perl confusion, nevermind...
> 
> Below is attached the log and ini files, and also how I run MTT.
> 
> Thanks,
> 
> Rafael
> 
> 
> client/mtt -d -v -p -f openmpi.ini 2>&1 |tee test.log
> 
> ...
> 
> 
> *** Run test phase starting
> >> Test run [trivial]
> Evaluating: trivial
> Found a match! trivial [trivial
> Evaluating: Simple
> >> Running with [openmpi-1.2.8] / [1.2.8] / [openmpi-1.2.8]
> Found MPI details: [mpi details: open mpi]
> Using [mpi details: open mpi] with [MPI Install: openmpi-1.2.8]
> Evaluating: # We can exit if the test passed or was skipped (i.e.,
> there's
>no need
># to cleanup).
>if test "$MTT_TEST_RUN_RESULT" = "passed" -o "$MTT_TEST_RUN_RESULT" =
>"skipped"; then
>exit 0
>fi
>
>if test "$MTT_TEST_HOSTFILE" != ""; then
>args="--hostfile $MTT_TEST_HOSTFILE"
>elif test "$MTT_TEST_HOSTLIST" != ""; then
>args="--host $MTT_TEST_HOSTLIST"
>fi
>orterun $args -np $MTT_TEST_NP --prefix $MTT_TEST_PREFIX
>mtt_ompi_cleanup.pl
> Got final exec: mpirun (_hostfile(), "("--hostfile ",
>"()")",   "(_hostlist(), "("--host ",
>"()")", "")") -np _np() --mca btl openib,self --debug
>--prefix _prefix() _executable() _argv()
> chdir /tmp/ompi-core-testers/installs/dLS2/tests/trivial/test_get__trivial
> Evaluating: require MTT::Test::Specify::Simple
> Evaluating: $ret = ::Test::Specify::Simple::Specify(@args)
> Evaluating: _executables(".")
> Got name: find_executables
> Got args: "."
> _do: $ret = MTT::Values::Functions::find_executables(".")
> _executables got .
> _exectuables returning: ./c_ring ./f77_ring ./f90_hello ./cxx_ring
>./f77_hello ./cxx_hello ./f90_ring ./c_hello
> *** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
> string ("8") as an ARRAY ref while "strict refs" in use at
> /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.

You see this in both the "trunk" and "ompi-core-testers"? I think this 
problem with array references was ironed out in the trunk.

-Ethan

> 
> On Wed, 2009-04-08 at 14:15 -0400, Ethan Mallove wrote:
> > On Wed, Apr/08/2009 11:36:05AM, Rafael Folco wrote:
> > > Well, I took a look at /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> > > 
> > > This piece of code looks wrong to me:
> > > 
> > >  if ($#{@$ret} < 0) {
> > > 
> > > $ret references an array
> > > @$ret points to the first element of this array
> > > $# returns the number of elements
> > > 
> > > So this line is trying to count elements of the first element??! Doesn't
> > > make sense. Correct me if I am wrong, what am I missing here ?
> > > 
> > > "if ($#{$ret} < 0) {" would be correct, without @. 
> > > 
> > > I believe "strict refs" has been forced somewhere on SLES11... I also
> > > tried on other distro and it works fine.
> > 
> > How do perl -V differ between the two distros?
> > 
> > I can not reproduce the error on a SLES 10 machine.
> > 
> > Could you run MTT with the --debug option and send the output with the
> > line 75 perl error? That might help me determine which INI param is
> > responsible for the error.
> > 
> > -Ethan
> > 
> > > 
> > > Thanks,
> > > 
> > > Rafael
> > > 
> > > On Tue, 2009-04-07 at 15:53 -0300, Rafael Folco wrote:
> > > > Hi,
> > > > 
> > > > I'm trying to run MTT on SLES11, but I am getting an error message
> > > > during the RUN phase and I can't figure out what is the problem.
> > > > 
> > > > *** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
> > > > string ("183") as an ARRAY ref while "strict refs" in use at
> > > > /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> > > > 
> > > > What I could see was that this error is nothing specific to any
> > > > particular test, it happens at certain points during the RUN phase.
> > > > Also, the BUILD phase has been completed successfully for all tests.
> > > > 
> > > > Anybody have already seen this? Any thoughts ?
> > > > 
> > > > Thanks i

Re: [MTT users] MTT Error on SLES11

[Ethan Mallove]

On Wed, Apr/08/2009 11:36:05AM, Rafael Folco wrote:
> Well, I took a look at /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> 
> This piece of code looks wrong to me:
> 
>  if ($#{@$ret} < 0) {
> 
> $ret references an array
> @$ret points to the first element of this array
> $# returns the number of elements
> 
> So this line is trying to count elements of the first element??! Doesn't
> make sense. Correct me if I am wrong, what am I missing here ?
> 
> "if ($#{$ret} < 0) {" would be correct, without @. 
> 
> I believe "strict refs" has been forced somewhere on SLES11... I also
> tried on other distro and it works fine.

How do perl -V differ between the two distros?

I can not reproduce the error on a SLES 10 machine.

Could you run MTT with the --debug option and send the output with the
line 75 perl error? That might help me determine which INI param is
responsible for the error.

-Ethan

> 
> Thanks,
> 
> Rafael
> 
> On Tue, 2009-04-07 at 15:53 -0300, Rafael Folco wrote:
> > Hi,
> > 
> > I'm trying to run MTT on SLES11, but I am getting an error message
> > during the RUN phase and I can't figure out what is the problem.
> > 
> > *** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
> > string ("183") as an ARRAY ref while "strict refs" in use at
> > /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> > 
> > What I could see was that this error is nothing specific to any
> > particular test, it happens at certain points during the RUN phase.
> > Also, the BUILD phase has been completed successfully for all tests.
> > 
> > Anybody have already seen this? Any thoughts ?
> > 
> > Thanks in advance.
> > 
> > Rafael
> > 
> > 
> 
> -- 
> Rafael Folco
> OpenHPC / Test Lead
> IBM Linux Technology Center
> E-Mail: rfo...@linux.vnet.ibm.com
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

[Ethan Mallove]

Hi Steve,

I see improvements in 1.3.1 as compared to 1.2.9 in Netpipe results.
The below Open MPI installations were compiled with the same compiler,
configure options, run on the same cluster, and run with the same MCA
parameters. (Note, ClusterTools 8.2 is essentially
1.3.1r20828.)

  http://www.open-mpi.org/mtt/index.php?do_redir=990

Though the above are just two selected Netpipe runs out of thousands
in the MTT database. It's possible with some digging, one could come
up with a couple runs showing 1.2.9 outperforming 1.3.1 (given a
certain set of MCA params, configure options, platform, etc).

-Ethan

On Mon, Apr/06/2009 09:33:46AM, Steve Kargl wrote:
> Hi,
> 
> I recently upgraded OpenMPI from 1.2.9 to 1.3 and then 1.3.1.
> One of my colleagues reported a dramatic drop in performance
> with one of his applications.  My investigation shows a factor
> of 10 drop in communication over the memory bus.  I've placed
> a figure that iilustrates the problem at 
> 
> http://troutmask.apl.washington.edu/~kargl/ompi_cmp.jpg
> 
> The legend in the figure has 'ver. 1.2.9  11 <--> 18'.  This
> means communication between node 11 and node 18 over GigE 
> ethernet in my cluster.  'ver. 1.2.9  20 <--> 20' means
> communication between processes on node 20 where node 20 has
> 8 processors.  The image clearly shows that communication over
> GigE is consistent among the versions of OpenMPI.  However, some
> change in going from 1.2.9 to 1.3.x is causing a drop in
> communication between processes on a single node.
> 
> Things to note.  Nodes 11, 18, and 20 are essentially idle
> before and after a test.  configure was run with the same set
> of options except with 1.3 and 1.3.1 I needed to disable ipv6:
> 
>   ./configure --prefix=/usr/local/openmpi-1.2.9 \
>--enable-orterun-prefix-by-default --enable-static
>--disable-shared
> 
>   ./configure --prefix=/usr/local/openmpi-1.3.1 \
>--enable-orterun-prefix-by-default --enable-static
>--disable-shared --disable-ipv6
> 
>   ./configure --prefix=/usr/local/openmpi-1.3.1 \
>--enable-orterun-prefix-by-default --enable-static
>--disable-shared --disable-ipv6
> 
> The operating system is FreeBSD 8.0 where nodes 18 and 20
> are quad-core, dual-cpu opteron based systems and node 11
> is a dual-core, dual-cpu opteron based system.  For additional
> information, I've placed the output of ompi_info at
> 
> http://troutmask.apl.washington.edu/~kargl/ompi_info-1.2.9
> http://troutmask.apl.washington.edu/~kargl/ompi_info-1.3.0
> http://troutmask.apl.washington.edu/~kargl/ompi_info-1.3.1
> 
> Any hints on tuning 1.3.1 would be appreciated?
> 
> -- 
> steve
> -- 
> Steve
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Linux opteron infiniband sunstudio configure problem

[Ethan Mallove]

On Mon, Mar/30/2009 07:05:25PM, Kevin McManus wrote:
> 
> > > I will try to reproduce the problem.
> > 
> > I am not able to reproduce this with openmpi-1.3.2a1r20880.tar.gz.
> > 
> >   $ uname -a
> >   Linux ... 2.6.16.46-0.12-smp #1 SMP Thu May 17 14:00:09 UTC 2007 x86_64 
> > x86_64 x86_64 GNU/Linux
> >   $ cc -V
> >   cc: Sun C 5.9 Linux_i386 Patch 124871-01 2007/07/31
> 
> interesting, this is a v.similar system
> 
> ->uname -a
> Linux ... 2.6.16.54-0.2.5-smp #1 SMP Mon Jan 21 13:29:51 UTC 2008 x86_64 
> x86_64 x86_64 GNU/Linux
> ->cc -V
> cc: Sun C 5.9 Linux_i386 Patch 124871-01 2007/07/31
> 
> I think we are looking in the wrong place, these syntax errors worry me,
> I have never seen a configure script throw one before.
> 
> ->ls -l /bin/sh
> lrwxrwxrwx 1 root root 4 2008-04-02 02:13 /bin/sh -> bash
> 
> So it is running in bash, which is unlikely to cause problems.

I do not think you will get different configure results launching
configure from different shells.

> Looking at the configure source it runs some near identical code for
> MCA basic before retiring ungracefully immediately after deciding
> that it can handle MCA bucket. Ungracefully hacking the offending
> parentheses from configure causes some v.strange things to happen. I
> will continue hacking but until this bears fruit... do you also need
> to include 'x86_64' as an argument for configure? What happens when
> you run 'uname -X'?

uname -X gives me "invalid option" on RHEL {4,5} and SLES {9,10}.
Post your config.log file.

-Ethan

> 
> TIA
> 
> 
> k.mcma...@gre.ac.uk - http://staffweb.cms.gre.ac.uk/~k.mcmanus
> --
> Dr Kevin McManusQueen Mary 413
> School of Computing & Mathematical Sciences,
> University of Greenwich,
> Old Royal Naval College,
> Park Row, Greenwich,   Tel +44 (0)208 331 8719 
> London, SE10 9LS   Fax +44 (0)208 331 8665
> --
> 
> University of Greenwich, a charity and company limited by
> guarantee, registered in England (reg no. 986729)
> Registered Office: Old Royal Naval College
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Compiling Open MPI with PGI compilers in 32-bit mode

[Ethan Mallove]

FYI - there is a Libtool thread/patch that resolved this issue:

  http://lists.gnu.org/archive/html/libtool/2009-03/msg00035.html

-Ethan

On Fri, Mar/20/2009 01:36:58PM, Ethan Mallove wrote:
> On Fri, Mar/20/2009 01:09:56PM, Ethan Mallove wrote:
> > Let me try this again. Below is the error from OMPI 1.3r20826. In my
> > last email, I accidentally posted the compiler error from Sun's
> > internal OMPI source repository.
> > 
> >   $ cd opal/mca/memory/ptmalloc2
> >   $ make
> >   /bin/sh ../../../../libtool --tag=CC   --mode=link pgcc  -O -DNDEBUG  
> > -tp=k8-32-export-dynamic-o libopenmpi-malloc.la -rpath 
> > /opt/SUNWhpc/HPC8.2/pgi/lib malloc.lo malloc-stats.lo  -lnsl -lutil  
> > -lpthread
> >   libtool: link: pgcc -shared  -fpic -DPIC  .libs/malloc.o 
> > .libs/malloc-stats.o   -lnsl -lutil -lpthread -lc-Wl,-soname 
> > -Wl,libopenmpi-malloc.so.0 -o .libs/libopenmpi-malloc.so.0.0.0
> >   /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o' is 
> > incompatible with i386:x86-64 output
> >   /usr/bin/ld: warning: i386 architecture of input file 
> > `.libs/malloc-stats.o' is incompatible with i386:x86-64 output
> >   .libs/malloc.o(.text+0x14d3): In function `munmap_chunk':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x1550): In function `mremap_chunk':
> >   : undefined reference to `opal_mem_hooks_release_hook'
> >   .libs/malloc.o(.text+0x2bd2): In function `_int_free':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x309e): In function `_int_realloc':
> >   : undefined reference to `opal_mem_hooks_release_hook'
> >   .libs/malloc.o(.text+0x3c1a): In function `opal_mem_free_ptmalloc2_sbrk':
> >   : undefined reference to `opal_mem_hooks_release_hook'
> >   .libs/malloc.o(.text+0x3f9b): In function `ptmalloc_init':
> >   : undefined reference to `opal_mem_hooks_set_support'
> >   .libs/malloc.o(.text+0x409d): In function `new_heap':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x40c5): In function `new_heap':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x413f): In function `new_heap':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x4188): In function `new_heap':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x4272): In function `heap_trim':
> >   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
> >   .libs/malloc.o(.text+0x449a): In function `arena_get2':
> >   : undefined reference to `opal_atomic_wmb'
> >   make: *** [libopenmpi-malloc.la] Error 2
> 
> I think we have located the problem.  Libtool has stripped out
> the -tp=k8-32 flag in the link step:
> 
>   /bin/sh ../../../../libtool --tag=CC --mode=link pgcc -O -DNDEBUG -tp=k8-32 
> -export-dynamic -o libopenmpi-malloc.la -rpath /opt/SUNWhpc/HPC8.2/pgi/lib 
> malloc.lo malloc-stats.lo -lnsl -lutil -lpthread
>   libtool: link: pgcc -shared -fpic -DPIC .libs/malloc.o .libs/malloc-stats.o 
> -lnsl -lutil -lpthread -lc -Wl,-soname -Wl,libopenmpi-malloc.so.0 -o 
> .libs/libopenmpi-malloc.so.0.0.0
> 
> If I add it back in manually, all is well:
> 
>   $ pgcc -tp=k8-32 -shared -fpic -DPIC .libs/malloc.o .libs/malloc-stats.o 
> -lnsl -lutil -lpthread -lc -Wl,-soname -Wl,libopenmpi-malloc.so.0 -o 
> .libs/libopenmpi-malloc.so.0.0.0
> 
> I am guessing there is some macro (LDFLAGS?) that I can override to
> prevent Libtool from stripping out the -tp=k8-32 flag.
> 
> -Ethan
> 
> 
> > 
> > -Ethan
> > 
> > 
> > On Fri, Mar/20/2009 12:49:06PM, Ethan Mallove wrote:
> > > Hi,
> > > 
> > > Has anyone successfully compiled Open MPI with the PGI compilers in
> > > 32-bit mode (e.g., using -tp=k8-32 flag)?  I am getting the following
> > > error with 32-bit:
> > > 
> > >$ cd opal/mca/memory/ptmalloc2
> > >$ make
> > >/bin/sh ../../../../libtool --tag=CC --mode=link pgcc -O -DNDEBUG 
> > > -tp=k8-32 -export-dynamic -o libopenmpi-malloc.la -rpath 
> > > /opt/SUNWhpc/HPC8.2/pgi/lib malloc.lo malloc-stats.lo -lnsl -lutil 
> > > -lpthread
> > >libtool: link: pgcc -shared -fpic -DPIC .libs/malloc.o 
> > > .libs/malloc-stats.o -lnsl -lutil -lpthread -lc -Wl,-soname 
> > > -Wl,libopenmpi-malloc.so.0 -o .libs/libopenmpi-malloc.so.0.0.0
> > >   /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o' 
> > > is incompatible w

Re: [OMPI users] Linux opteron infiniband sunstudio configure problem

[Ethan Mallove]

On Mon, Mar/30/2009 09:04:26AM, Ethan Mallove wrote:
> On Thu, Mar/26/2009 04:52:28PM, Kevin McManus wrote:
> > 
> > Hi All,
> > 
> > As a complete beginner (to OpenMPI) I am attempting to build on
> > a Linux opteron infiniband platform using SunStudio compilers.
> > 
> > My build script looks like...
> > 
> > #!/bin/sh
> > 
> > ../configure x86_64 \
> > CC=cc CXX=CC F77=f77 FC=f90 \
> > CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 \
> > --enable-static \
> > --enable-orterun-prefix-by-default \
> > --enable-mpi-f90 \
> > --enable-mpi-threads \
> > --with-psm
> 
> We configure with --disable-mpi-threads, and have not tested
> --with-psm and --with-static. Do you provide --with-psm a DIR
> argument? "configure --help" shows: 
> 
>   ...
>   --with-psm(=DIR)Build PSM (QLogic InfiniPath PSM) support, searching
>   for libraries in DIR
>   --with-psm-libdir=DIR   Search for PSM (QLogic InfiniPath PSM) libraries in
>   DIR
>   ...
>  
> I will try to reproduce the problem.

I am not able to reproduce this with openmpi-1.3.2a1r20880.tar.gz.

  $ uname -a
  Linux ... 2.6.16.46-0.12-smp #1 SMP Thu May 17 14:00:09 UTC 2007 x86_64 
x86_64 x86_64 GNU/Linux
  $ cc -V
  cc: Sun C 5.9 Linux_i386 Patch 124871-01 2007/07/31

-Ethan

> 
> -Ethan
> 
> 
> > 
> > Not sure why I need to put x86_64 as the first argument, just a lucky
> > guess after... 
> > 
> > configure: error: cannot guess build type; you must specify one
> > 
> > ...but I hit another problem...
> > 
> > --- MCA component allocator:basic (no configuration)
> > checking for MCA component allocator:basic compile mode... static
> > checking if MCA component allocator:basic can compile... yes
> > 
> > --- MCA component allocator:bucket (no configuration)
> > checking for MCA component allocator:bucket compile mode... static
> > checking if MCA component allocator:bucket can compile... yes
> > ../configure: line 123247: syntax error near unexpected token `('
> > ../configure: line 123247: `{ $as_echo "$as_me:$LINENO: 
> > WARNING: This component was selected as the default (direct call)" >&5'
> > 
> > Have no idea of why I should need an MCA bucket but it seems
> > to trigger a problem. If I hack out the parentheses I end up
> > hitting more parenthetic syntax errors.
> > 
> > Any suggestions gratefully received.
> > 
> > 
> > k.mcma...@gre.ac.uk - http://staffweb.cms.gre.ac.uk/~k.mcmanus
> > --
> > Dr Kevin McManusQueen Mary 413
> > School of Computing & Mathematical Sciences,
> > University of Greenwich,
> > Old Royal Naval College,
> > Park Row, Greenwich,   Tel +44 (0)208 331 8719 
> > London, SE10 9LS   Fax +44 (0)208 331 8665
> > --
> > 
> > University of Greenwich, a charity and company limited by
> > guarantee, registered in England (reg no. 986729)
> > Registered Office: Old Royal Naval College
> > ___
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Linux opteron infiniband sunstudio configure problem

[Ethan Mallove]

On Thu, Mar/26/2009 04:52:28PM, Kevin McManus wrote:
> 
> Hi All,
> 
> As a complete beginner (to OpenMPI) I am attempting to build on
> a Linux opteron infiniband platform using SunStudio compilers.
> 
> My build script looks like...
> 
> #!/bin/sh
> 
> ../configure x86_64 \
> CC=cc CXX=CC F77=f77 FC=f90 \
> CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 \
> --enable-static \
> --enable-orterun-prefix-by-default \
> --enable-mpi-f90 \
> --enable-mpi-threads \
> --with-psm

We configure with --disable-mpi-threads, and have not tested
--with-psm and --with-static. Do you provide --with-psm a DIR
argument? "configure --help" shows: 

  ...
  --with-psm(=DIR)Build PSM (QLogic InfiniPath PSM) support, searching
  for libraries in DIR
  --with-psm-libdir=DIR   Search for PSM (QLogic InfiniPath PSM) libraries in
  DIR
  ...

I will try to reproduce the problem.

-Ethan


> 
> Not sure why I need to put x86_64 as the first argument, just a lucky
> guess after... 
> 
> configure: error: cannot guess build type; you must specify one
> 
> ...but I hit another problem...
> 
> --- MCA component allocator:basic (no configuration)
> checking for MCA component allocator:basic compile mode... static
> checking if MCA component allocator:basic can compile... yes
> 
> --- MCA component allocator:bucket (no configuration)
> checking for MCA component allocator:bucket compile mode... static
> checking if MCA component allocator:bucket can compile... yes
> ../configure: line 123247: syntax error near unexpected token `('
> ../configure: line 123247: `{ $as_echo "$as_me:$LINENO: WARNING: 
> This component was selected as the default (direct call)" >&5'
> 
> Have no idea of why I should need an MCA bucket but it seems
> to trigger a problem. If I hack out the parentheses I end up
> hitting more parenthetic syntax errors.
> 
> Any suggestions gratefully received.
> 
> 
> k.mcma...@gre.ac.uk - http://staffweb.cms.gre.ac.uk/~k.mcmanus
> --
> Dr Kevin McManusQueen Mary 413
> School of Computing & Mathematical Sciences,
> University of Greenwich,
> Old Royal Naval College,
> Park Row, Greenwich,   Tel +44 (0)208 331 8719 
> London, SE10 9LS   Fax +44 (0)208 331 8665
> --
> 
> University of Greenwich, a charity and company limited by
> guarantee, registered in England (reg no. 986729)
> Registered Office: Old Royal Naval College
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Compiling Open MPI with PGI compilers in 32-bit mode

[Ethan Mallove]

On Fri, Mar/20/2009 11:06:00AM, Doug Reeder wrote:
>Ethan,
>It looks likesome of the object files that you are trying to link to the
>malloc.o and malloc-stats.o were compiled as 64 bit objects. Are you using
>the 32 bit compiler flag for the compile step as well as the link step.

It looks like I'm in 32-bit mode for the compile:

  libtool: compile:  pgcc -DHAVE_CONFIG_H -I. -I../../opal/include 
-I../../orte/include -I../../ompi/include 
-I../../opal/mca/paffinity/linux/plpa/src/libplpa -I../.. -D_REENTRANT -O 
-DNDEBUG -tp=k8-32 -c malloc.c  -fpic -DPIC -o .libs/malloc.o

  libtool: compile:  pgcc -DHAVE_CONFIG_H -I. -I../../../../opal/include 
-I../../../../orte/include -I../../../../ompi/include 
-I../../../../opal/mca/paffinity/linux/plpa/src/libplpa -DMALLOC_DEBUG=0 
-D_GNU_SOURCE=1 -DUSE_TSD_DATA_HACK=1 -DMALLOC_HOOKS=1 -I./sysdeps/pthread 
-I./sysdeps/generic -I../../../.. -D_REENTRANT -O -DNDEBUG -tp=k8-32 -c 
malloc-stats.c  -fpic -DPIC -o .libs/malloc-stats.o

Can't I ignore those warnings for now?  It seems PGI allows me to
create a working executable, even if I mix "i386" with "i386:x86-64":

  $ pgcc -c -tp=k8-32 bar.c -o bar_32.o
  $ pgcc -tp=k8-64 foo.c bar_32.o -o foo_mix
  /home/em162155/tmp/foo.c:
  /usr/bin/ld: warning: i386 architecture of input file `bar_32.o' is 
incompatible with i386:x86-64 output
  $ ./foo_mix
  foo

-Ethan


>Doug Reeder
>    On Mar 20, 2009, at 10:49 AM, Ethan Mallove wrote:
> 
>  Hi,
> 
>  Has anyone successfully compiled Open MPI with the PGI compilers in
>  32-bit mode (e.g., using -tp=k8-32 flag)?  I am getting the following
>  error with 32-bit:
> 
>$ cd opal/mca/memory/ptmalloc2
>$ make
>/bin/sh ../../../../libtool --tag=CC --mode=link pgcc -O -DNDEBUG
>  -tp=k8-32 -export-dynamic -o libopenmpi-malloc.la -rpath
>  /opt/SUNWhpc/HPC8.2/pgi/lib malloc.lo malloc-stats.lo -lnsl -lutil
>  -lpthread
>libtool: link: pgcc -shared -fpic -DPIC .libs/malloc.o
>  .libs/malloc-stats.o -lnsl -lutil -lpthread -lc -Wl,-soname
>  -Wl,libopenmpi-malloc.so.0 -o .libs/libopenmpi-malloc.so.0.0.0
>   /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o'
>  is incompatible with i386:x86-64 output
>   /usr/bin/ld: warning: i386 architecture of input file
>  `.libs/malloc-stats.o' is incompatible with i386:x86-64 output
>   .libs/malloc.o(.text+0xcb3): In function `realloc_check':
>   : undefined reference to `opal_memcpy_base_module'
>   .libs/malloc.o(.text+0x14e3): In function `munmap_chunk':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x1560): In function `mremap_chunk':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x2be2): In function `_int_free':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x30ae): In function `_int_realloc':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x3c2a): In function
>  `opal_mem_free_ptmalloc2_sbrk':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x3fab): In function `ptmalloc_init':
>   : undefined reference to `opal_mem_hooks_set_support'
>   .libs/malloc.o(.text+0x40ad): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x40d5): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x414f): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x4198): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x4282): In function `heap_trim':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x44aa): In function `arena_get2':
>   : undefined reference to `opal_atomic_wmb'
>   make: *** [libopenmpi-malloc.la] Error 2
> 
>  -Ethan
>  ___
>  users mailing list
>  us...@open-mpi.org
>  http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> References
> 
>Visible links
>. mailto:us...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/users

> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Compiling Open MPI with PGI compilers in 32-bit mode

[Ethan Mallove]

Let me try this again. Below is the error from OMPI 1.3r20826. In my
last email, I accidentally posted the compiler error from Sun's
internal OMPI source repository.

  $ cd opal/mca/memory/ptmalloc2
  $ make
  /bin/sh ../../../../libtool --tag=CC   --mode=link pgcc  -O -DNDEBUG  
-tp=k8-32-export-dynamic-o libopenmpi-malloc.la -rpath 
/opt/SUNWhpc/HPC8.2/pgi/lib malloc.lo malloc-stats.lo  -lnsl -lutil  -lpthread
  libtool: link: pgcc -shared  -fpic -DPIC  .libs/malloc.o .libs/malloc-stats.o 
  -lnsl -lutil -lpthread -lc-Wl,-soname -Wl,libopenmpi-malloc.so.0 -o 
.libs/libopenmpi-malloc.so.0.0.0
  /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o' is 
incompatible with i386:x86-64 output
  /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc-stats.o' 
is incompatible with i386:x86-64 output
  .libs/malloc.o(.text+0x14d3): In function `munmap_chunk':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x1550): In function `mremap_chunk':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x2bd2): In function `_int_free':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x309e): In function `_int_realloc':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x3c1a): In function `opal_mem_free_ptmalloc2_sbrk':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x3f9b): In function `ptmalloc_init':
  : undefined reference to `opal_mem_hooks_set_support'
  .libs/malloc.o(.text+0x409d): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x40c5): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x413f): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x4188): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x4272): In function `heap_trim':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x449a): In function `arena_get2':
  : undefined reference to `opal_atomic_wmb'
  make: *** [libopenmpi-malloc.la] Error 2

-Ethan


On Fri, Mar/20/2009 12:49:06PM, Ethan Mallove wrote:
> Hi,
> 
> Has anyone successfully compiled Open MPI with the PGI compilers in
> 32-bit mode (e.g., using -tp=k8-32 flag)?  I am getting the following
> error with 32-bit:
> 
>$ cd opal/mca/memory/ptmalloc2
>$ make
>/bin/sh ../../../../libtool --tag=CC --mode=link pgcc -O -DNDEBUG 
> -tp=k8-32 -export-dynamic -o libopenmpi-malloc.la -rpath 
> /opt/SUNWhpc/HPC8.2/pgi/lib malloc.lo malloc-stats.lo -lnsl -lutil -lpthread
>libtool: link: pgcc -shared -fpic -DPIC .libs/malloc.o 
> .libs/malloc-stats.o -lnsl -lutil -lpthread -lc -Wl,-soname 
> -Wl,libopenmpi-malloc.so.0 -o .libs/libopenmpi-malloc.so.0.0.0
>   /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o' is 
> incompatible with i386:x86-64 output
>   /usr/bin/ld: warning: i386 architecture of input file 
> `.libs/malloc-stats.o' is incompatible with i386:x86-64 output
>   .libs/malloc.o(.text+0xcb3): In function `realloc_check':
>   : undefined reference to `opal_memcpy_base_module'
>   .libs/malloc.o(.text+0x14e3): In function `munmap_chunk':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x1560): In function `mremap_chunk':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x2be2): In function `_int_free':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x30ae): In function `_int_realloc':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x3c2a): In function `opal_mem_free_ptmalloc2_sbrk':
>   : undefined reference to `opal_mem_hooks_release_hook'
>   .libs/malloc.o(.text+0x3fab): In function `ptmalloc_init':
>   : undefined reference to `opal_mem_hooks_set_support'
>   .libs/malloc.o(.text+0x40ad): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x40d5): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x414f): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x4198): In function `new_heap':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x4282): In function `heap_trim':
>   : undefined reference to `opal_mem_free_ptmalloc2_munmap'
>   .libs/malloc.o(.text+0x44aa): In function `arena_get2':
>   : undefined reference to `opal_atomic_wmb'
>   make: *** [libopenmpi-malloc.la] Error 2
> 
> -Ethan
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

[OMPI users] Compiling Open MPI with PGI compilers in 32-bit mode

[Ethan Mallove]

Hi,

Has anyone successfully compiled Open MPI with the PGI compilers in
32-bit mode (e.g., using -tp=k8-32 flag)?  I am getting the following
error with 32-bit:

   $ cd opal/mca/memory/ptmalloc2
   $ make
   /bin/sh ../../../../libtool --tag=CC --mode=link pgcc -O -DNDEBUG -tp=k8-32 
-export-dynamic -o libopenmpi-malloc.la -rpath /opt/SUNWhpc/HPC8.2/pgi/lib 
malloc.lo malloc-stats.lo -lnsl -lutil -lpthread
   libtool: link: pgcc -shared -fpic -DPIC .libs/malloc.o .libs/malloc-stats.o 
-lnsl -lutil -lpthread -lc -Wl,-soname -Wl,libopenmpi-malloc.so.0 -o 
.libs/libopenmpi-malloc.so.0.0.0
  /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc.o' is 
incompatible with i386:x86-64 output
  /usr/bin/ld: warning: i386 architecture of input file `.libs/malloc-stats.o' 
is incompatible with i386:x86-64 output
  .libs/malloc.o(.text+0xcb3): In function `realloc_check':
  : undefined reference to `opal_memcpy_base_module'
  .libs/malloc.o(.text+0x14e3): In function `munmap_chunk':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x1560): In function `mremap_chunk':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x2be2): In function `_int_free':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x30ae): In function `_int_realloc':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x3c2a): In function `opal_mem_free_ptmalloc2_sbrk':
  : undefined reference to `opal_mem_hooks_release_hook'
  .libs/malloc.o(.text+0x3fab): In function `ptmalloc_init':
  : undefined reference to `opal_mem_hooks_set_support'
  .libs/malloc.o(.text+0x40ad): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x40d5): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x414f): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x4198): In function `new_heap':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x4282): In function `heap_trim':
  : undefined reference to `opal_mem_free_ptmalloc2_munmap'
  .libs/malloc.o(.text+0x44aa): In function `arena_get2':
  : undefined reference to `opal_atomic_wmb'
  make: *** [libopenmpi-malloc.la] Error 2

-Ethan

Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

On Tue, Jan/06/2009 10:33:31AM, Ethan Mallove wrote:
> On Mon, Jan/05/2009 10:14:30PM, Brian Barrett wrote:
> > Sorry I haven't jumped in this thread earlier -- I've been a bit behind.
> >
> > The multi-lib support worked at one time, and I can't think of why it would 
> > have changed.  The one condition is that libdir, includedir, etc. *MUST* be 
> > specified relative to $prefix for it to work.  It looks like you were 
> > defining them as absolute paths, so you'd have to set libdir directly, 
> > which will never work in multi-lib because mpirun and the app likely have 
> > different word sizes and therefore different libdirs.  
> >
> 
> I see. I'll try configuring with relative paths using ${prefix} and
> the like.
> 
> > More information is on the multilib page in the wiki:
> >
> >   https://svn.open-mpi.org/trac/ompi/wiki/MultiLib
> >
> 
> I removed this line from the MultiLib wiki page since Open MPI *is*
> now relocatable using the OPAL_PREFIX env vars:
> 
>   "Presently, Open MPI is not relocatable.  That is, Open MPI *must*
>   be installed and executed from which ever prefix was specified
>   during configure.  This is planned to change in the very near
>   future."
> 
> Thanks,
> Ethan
> 
> 
> > There is actually one condition we do not handle properly, the prefix flag 
> > to mpirun.  The LD_LIBRARY_PATH will only be set for the word size of 
> > mpirun, and not the executable.  Really, both would have to be added (so 
> > that both orted, which is likely always 32 bit in a multilib situation and 
> > the app both find their libraries).
> >
> > Brian
> >
> > On Jan 5, 2009, at 6:02 PM, Jeff Squyres wrote:
> >
> >> I honestly haven't thought through the ramifications of doing a multi-lib 
> >> build with OPAL_PREFIX et al. :-\
> >>
> >> If you setenv OPAL_LIBDIR, it'll use whatever you set it to, so it doesn't 
> >> matter what you configured --libdir with.  Additionally 
> >> mca/installdirs/config/install_dirs.h has this by default:
> >>
> >> #define OPAL_LIBDIR "${exec_prefix}/lib"
> >>
> >> Hence, if you use a default --libdir and setenv OPAL_PREFIX, then the 
> >> libdir should pick up the right thing (because it's based on the prefix).  
> >> But if you use --libdir that is *not* based on ${exec_prefix}, then you 
> >> might run into problems.
> >>
> >> Perhaps you can '--libdir="${exec_prefix}/lib64"' so that you can have 
> >> your custom libdir, but still have it dependent upon the prefix that gets 
> >> expanded at run time...?


Can the Open MPI configure setup handle ${exec_prefix} at the command
line? ${exec_prefix} seems to be getting eval'd to "NONE" in the
sub-configure's, and I get the following error:

  ...
  *** GNU libltdl setup
  configure: OMPI configuring in opal/libltdl
  configure: running /bin/bash './configure'  'CC=cc' 'CXX=CC' 'F77=f77' 
'FC=f90' '--without-threads' '--enable-heterogeneous' '--enable-cxx-exceptions' 
'--enable-shared' '--enable-orterun-prefix-by-default' '--with-sge' 
'--enable-mpi-f90' '--with-mpi-f90-size=small' '--disable-mpi-threads' 
'--disable-progress-threads' '--disable-debug' 'CFLAGS=-xtarget=ultra3 -m32 
-xarch=sparcvis2 -xprefetch -xprefetch_level=2 -xvector=lib -xdepend=yes 
-xbuiltin=%all -xO5' 'CXXFLAGS=-xtarget=ultra3 -m32 -xarch=sparcvis2 -xprefetch 
-xprefetch_level=2 -xvector=lib -xdepend=yes -xbuiltin=%all -xO5' 
'FFLAGS=-xtarget=ultra3 -m32 -xarch=sparcvis2 -xprefetch -xprefetch_level=2 
-xvector=lib -stackvar -xO5' 'FCFLAGS=-xtarget=ultra3 -m32 -xarch=sparcvis2 
-xprefetch -xprefetch_level=2 -xvector=lib -stackvar -xO5' 
'--prefix=/opt/SUNWhpc/HPC8.2/sun' '--libdir=NONE/lib' 
'--includedir=/opt/SUNWhpc/HPC8.2/sun/include' '--without-mx' 
'--with-tm=/ws/ompi-tools/orte/torque/current/shared-install32' 
'--with-contrib-vt-flags=--prefix=/opt/SUNWhpc/HPC8.2/sun --libdir='/lib' 
--includedir='/include' LDFLAGS=-R/opt/SUNWhpc/HPC8.2/sun/lib' 
'--with-package-string=ClusterTools 8.2' '--with-ident-string=@(#)RELEASE 
VERSION 1.3r20204-ct8.2-b01b-r10' --enable-ltdl-convenience 
--disable-ltdl-install --enable-shared --disable-static --cache-file=/dev/null 
--srcdir=. configure: WARNING: Unrecognized options: --without-threads, 
--enable-heterogeneous, --enable-cxx-exceptions, 
--enable-orterun-prefix-by-default, --with-sge, --enable-mpi-f90, 
--with-mpi-f90-size, --disable-mpi-threads, --disable-progress-threads, 
--disable-debug, --without-mx, --with-tm, --with-contrib-vt-flags, 
--with-package-string, --with-ident-string, --enable-ltdl-convenience
  configure: error: expected an absolute directory name for --libdir: NONE/lib
  configure: /bin/bash './c

Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

On Mon, Jan/05/2009 10:14:30PM, Brian Barrett wrote:
> Sorry I haven't jumped in this thread earlier -- I've been a bit behind.
>
> The multi-lib support worked at one time, and I can't think of why it would 
> have changed.  The one condition is that libdir, includedir, etc. *MUST* be 
> specified relative to $prefix for it to work.  It looks like you were 
> defining them as absolute paths, so you'd have to set libdir directly, 
> which will never work in multi-lib because mpirun and the app likely have 
> different word sizes and therefore different libdirs.  
>

I see. I'll try configuring with relative paths using ${prefix} and
the like.

> More information is on the multilib page in the wiki:
>
>   https://svn.open-mpi.org/trac/ompi/wiki/MultiLib
>

I removed this line from the MultiLib wiki page since Open MPI *is*
now relocatable using the OPAL_PREFIX env vars:

  "Presently, Open MPI is not relocatable.  That is, Open MPI *must*
  be installed and executed from which ever prefix was specified
  during configure.  This is planned to change in the very near
  future."

Thanks,
Ethan


> There is actually one condition we do not handle properly, the prefix flag 
> to mpirun.  The LD_LIBRARY_PATH will only be set for the word size of 
> mpirun, and not the executable.  Really, both would have to be added (so 
> that both orted, which is likely always 32 bit in a multilib situation and 
> the app both find their libraries).
>
> Brian
>
> On Jan 5, 2009, at 6:02 PM, Jeff Squyres wrote:
>
>> I honestly haven't thought through the ramifications of doing a multi-lib 
>> build with OPAL_PREFIX et al. :-\
>>
>> If you setenv OPAL_LIBDIR, it'll use whatever you set it to, so it doesn't 
>> matter what you configured --libdir with.  Additionally 
>> mca/installdirs/config/install_dirs.h has this by default:
>>
>> #define OPAL_LIBDIR "${exec_prefix}/lib"
>>
>> Hence, if you use a default --libdir and setenv OPAL_PREFIX, then the 
>> libdir should pick up the right thing (because it's based on the prefix).  
>> But if you use --libdir that is *not* based on ${exec_prefix}, then you 
>> might run into problems.
>>
>> Perhaps you can '--libdir="${exec_prefix}/lib64"' so that you can have 
>> your custom libdir, but still have it dependent upon the prefix that gets 
>> expanded at run time...?
>>
>> (again, I'm not thinking all of this through -- just offering a few 
>> suggestions off the top of my head that you'll need to test / trace the 
>> code to be sure...)
>>
>>
>> On Jan 5, 2009, at 1:35 PM, Ethan Mallove wrote:
>>
>>> On Thu, Dec/25/2008 08:12:49AM, Jeff Squyres wrote:
>>>> It's quite possible that we don't handle this situation properly.  Won't
>>>> you need to libdir's (one for the 32 bit OMPI executables, and one for 
>>>> the
>>>> 64 bit MPI apps)?
>>>
>>> I don't need an OPAL environment variable for the executables, just a
>>> single OPAL_LIBDIR var for the libraries. (One set of 32-bit
>>> executables runs with both 32-bit and 64-bit libraries.) I'm guessing
>>> OPAL_LIBDIR will not work for you if you configure with a non-standard
>>> --libdir option.
>>>
>>> -Ethan
>>>
>>>
>>>>
>>>> On Dec 23, 2008, at 3:58 PM, Ethan Mallove wrote:
>>>>
>>>>> I think the problem is that I am doing a multi-lib build. I have
>>>>> 32-bit libraries in lib/, and 64-bit libraries in lib/64. I assume I
>>>>> do not see the issue for 32-bit tests, because all the dependencies
>>>>> are where Open MPI expects them to be. For the 64-bit case, I tried
>>>>> setting OPAL_LIBDIR to /opt/openmpi-relocated/lib/lib64, but no luck.
>>>>> Given the below configure arguments, what do my OPAL_* env vars need
>>>>> to be? (Also, could using --enable-orterun-prefix-by-default interfere
>>>>> with OPAL_PREFIX?)
>>>>>
>>>>>  $ ./configure CC=cc CXX=CC F77=f77 FC=f90  --with-openib
>>>>> --without-udapl --disable-openib-ibcm --enable-heterogeneous
>>>>> --enable-cxx-exceptions --enable-shared 
>>>>> --enable-orterun-prefix-by-default
>>>>> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small
>>>>> --disable-mpi-threads --disable-progress-threads   --disable-debug
>>>>> CFLAGS="-m32 -xO5" CXXFLAGS="-m32 -xO5" FFLAGS="-m32 -xO5"  
>>>>> FCFLAGS="-m32
>>>>> -xO5"
>>>>> --prefix=/workspace/em16

Re: [MTT users] Patch for whatami (new sles10 lsb-release fileformat)

[Ethan Mallove]

On Mon, Jan/05/2009 07:36:06PM, Jeff Squyres wrote:
> Ethan -- did we ever reply to Brian about this?
>

Oops. I did not.

> Do you have any thoughts on the issue?

My thought is that I definitely prefer the short string (rhel) to the
new long one (RedHatEnterprise), and I'm less concerned about
whatami's efficiency. In other words, "a". (I generally only run
whatami a few times over the course of a long MTT run.)

-Ethan


>
>
> On Nov 7, 2008, at 3:07 PM, Brian Elliott Finley wrote:
>
>> Well, we don't want to be changing the results that folks are already
>> relying on, but it looks like we are.
>>
>> The benefit in putting the lsb test at the top is that most new and
>> future distros can be identified by that.
>>
>> But, it looks like the safest thing to do is to put it at the bottom.
>> Unfortunately, that means that we have to run through all the other
>> tests, which will not match, for all future distros.  Maybe it happens
>> so fast, even with the extra tests, that it's not worth worrying about.
>>
>> Options:
>>  a) Just put the lsb tests at the end (more computer effort over time)
>>  b) Put the lsb tests in the middle, after distro tests that are
>> already defined, but that we know also have lsb info (more people effort)
>>  c) I don't think leaving as is is an option, because if someone
>> updates to the latest version of whatami, it may break existing tests
>> they depend on.
>>
>> I'm leaning towards a).
>>
>> Thoughts?
>>
>> -Brian
>>
>>
>> Jeff Squyres wrote:
>>> Ditto:
>>>
>>> [8:23] svbu-mpi:~/svn/mtt % ./client/whatami/whatami
>>> linux-RedHatEnterpriseAS_4-x86_64
>>>
>>>
>>> On Nov 7, 2008, at 11:16 AM, Ethan Mallove wrote:
>>>
>>>> On Thu, Nov/06/2008 04:24:15PM, Jeff Squyres wrote:
>>>>> Thanks Brian.  Ethan -- if it works for you, can you pull down a new
>>>>> version and commit to MTT's SVN?
>>>>
>>>> Done.
>>>>
>>>> I get "linux-RedHatEnterpriseServer_5-ia32", instead of
>>>> "linux-rhel5-ia32" now. I guess I sort of prefer the short
>>>> version. Oh well.
>>>>
>>>> -Ethan
>>>>
>>>>>
>>>>>
>>>>> On Nov 6, 2008, at 3:19 PM, Brian Elliott Finley wrote:
>>>>>
>>>>>> All,
>>>>>>
>>>>>> I've committed a new version that includes the version referenced
>>>>>> below,
>>>>>> and changed the $() to ``.
>>>>>>
>>>>>> Cheers, -Brian
>>>>>>
>>>>>>
>>>>>> Tim Mattox wrote:
>>>>>>> The 2008.10.31 version that Jeff forwarded works fine on SLES9 PPC64.
>>>>>>>
>>>>>>> 2008/10/31 Jeff Squyres <jsquy...@cisco.com>:
>>>>>>>> Sorry; my stupid mailer didn't display it as an attachment when I
>>>>>>>> forwarded.
>>>>>>>> Try this.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Oct 31, 2008, at 1:56 PM, Ethan Mallove wrote:
>>>>>>>>
>>>>>>>>> On Fri, Oct/31/2008 01:01:02PM, Jeff Squyres wrote:
>>>>>>>>>> Re-sending to the MTT list, because I think our listserver will
>>>>>>>>>> automatically block Brian (I'll add him to the whitelist for 
>>>>>>>>>> future
>>>>>>>>>> posts!).
>>>>>>>>> I'm getting a syntax error:
>>>>>>>>>
>>>>>>>>> $ whatami
>>>>>>>>> whatami: syntax error at line 149:  `DISTRIB_ID=$' unexpected
>>>>>>>>>
>>>>>>>>> Not sure if I'm doing a bad copy/paste out of my mail
>>>>>>>>> client. Jeff, can you send the new whatami script as an
>>>>>>>>> attachment?
>>>>>>>>>
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Jeff Squyres
>>>>>>>> Cisco Systems
>>>>>>>>
>>>>>>>> ___
>>>>>>>> mtt-users mailing list
>>>>>>>> mtt-us...@open-mpi.org
>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Brian Elliott Finley
>>>>>> CIS / Argonne National Laboratory
>>>>>> Office: 630.252.4742
>>>>>> Mobile: 630.631.6621
>>>>>> ___
>>>>>> mtt-users mailing list
>>>>>> mtt-us...@open-mpi.org
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>
>>>>>
>>>>> -- 
>>>>> Jeff Squyres
>>>>> Cisco Systems
>>>>>
>>>>> ___
>>>>> mtt-users mailing list
>>>>> mtt-us...@open-mpi.org
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>> ___
>>>> mtt-users mailing list
>>>> mtt-us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>
>>>
>>
>> -- 
>> Brian Elliott Finley
>> CIS / Argonne National Laboratory
>> Office: 630.252.4742
>> Mobile: 630.631.6621
>
>
> -- 
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

On Thu, Dec/25/2008 08:12:49AM, Jeff Squyres wrote:
> It's quite possible that we don't handle this situation properly.  Won't 
> you need to libdir's (one for the 32 bit OMPI executables, and one for the 
> 64 bit MPI apps)?

I don't need an OPAL environment variable for the executables, just a
single OPAL_LIBDIR var for the libraries. (One set of 32-bit
executables runs with both 32-bit and 64-bit libraries.) I'm guessing
OPAL_LIBDIR will not work for you if you configure with a non-standard
--libdir option.

-Ethan


>
> On Dec 23, 2008, at 3:58 PM, Ethan Mallove wrote:
>
>> I think the problem is that I am doing a multi-lib build. I have
>> 32-bit libraries in lib/, and 64-bit libraries in lib/64. I assume I
>> do not see the issue for 32-bit tests, because all the dependencies
>> are where Open MPI expects them to be. For the 64-bit case, I tried
>> setting OPAL_LIBDIR to /opt/openmpi-relocated/lib/lib64, but no luck.
>> Given the below configure arguments, what do my OPAL_* env vars need
>> to be? (Also, could using --enable-orterun-prefix-by-default interfere
>> with OPAL_PREFIX?)
>>
>>$ ./configure CC=cc CXX=CC F77=f77 FC=f90  --with-openib 
>> --without-udapl --disable-openib-ibcm --enable-heterogeneous 
>> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default 
>> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small 
>> --disable-mpi-threads --disable-progress-threads   --disable-debug  
>> CFLAGS="-m32 -xO5" CXXFLAGS="-m32 -xO5" FFLAGS="-m32 -xO5"  FCFLAGS="-m32 
>> -xO5" 
>> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
>>  
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>>  
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib
>>  
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include
>>  
>> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install32 
>> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v!
>> 20z-12/ompi-tarball-testing/installs/DGQx/install 
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>>  
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib
>>  
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include
>>  
>> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib"
>>
>>$ ./confgiure CC=cc CXX=CC F77=f77 FC=f90  --with-openib 
>> --without-udapl --disable-openib-ibcm --enable-heterogeneous 
>> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default 
>> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small 
>> --disable-mpi-threads --disable-progress-threads   --disable-debug  
>> CFLAGS="-m64 -xO5" CXXFLAGS="-m64 -xO5" FFLAGS="-m64 -xO5"  FCFLAGS="-m64 
>> -xO5" 
>> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
>>  
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>>  
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
>>  
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
>>  
>> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install64 
>> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/!
>> burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install 
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>>  
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
>>  
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
>>  
>> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib"
>>  
>> --disable-binaries
>>
>> -Ethan
>>
>>>
>>>
>>> On Dec 22, 2008, at 12:42 PM, Ethan Mallove wrote:
>>>
>>>> Can anyone get OPAL_PREFIX to wo

Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

ing/installs/DGQx/install/man
 
--libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
 
--includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
 --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install64 
--with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
 
--mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
 
--libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
 
--includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
 
LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib"
 --disable-binaries

-Ethan

>
>
> On Dec 22, 2008, at 12:42 PM, Ethan Mallove wrote:
>
>> Can anyone get OPAL_PREFIX to work on Linux? A simple test is to see
>> if the following works for any mpicc/mpirun:
>>
>>  $ mv  /tmp/foo
>>  $ set OPAL_PREFIX /tmp/foo
>>  $ mpicc ...
>>  $ mpirun ...
>>
>> If you are able to get the above to run successfully, I'm interested
>> in your config.log file.
>>
>> Thanks,
>> Ethan
>>
>>
>> On Thu, Dec/18/2008 11:03:25AM, Ethan Mallove wrote:
>>> Hello,
>>>
>>> The below FAQ lists instructions on how to use a relocated Open MPI
>>> installation:
>>>
>>>  http://www.open-mpi.org/faq/?category=building#installdirs
>>>
>>> On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work for
>>> me with both MPI (hello_c) and non-MPI (hostname) programs. On Linux,
>>> I can only get the non-MPI case to work. Here are the environment
>>> variables I am setting:
>>>
>>>  $ cat setenv_opal_prefix.csh
>>>  set opal_prefix = "/opt/openmpi-relocated"
>>>
>>>  setenv OPAL_PREFIX $opal_prefix
>>>  setenv OPAL_BINDIR $opal_prefix/bin
>>>  setenv OPAL_SBINDIR$opal_prefix/sbin
>>>  setenv OPAL_DATAROOTDIR$opal_prefix/share
>>>  setenv OPAL_SYSCONFDIR $opal_prefix/etc
>>>  setenv OPAL_SHAREDSTATEDIR $opal_prefix/com
>>>  setenv OPAL_LOCALSTATEDIR  $opal_prefix/var
>>>  setenv OPAL_LIBDIR $opal_prefix/lib
>>>  setenv OPAL_INCLUDEDIR $opal_prefix/include
>>>  setenv OPAL_INFODIR$opal_prefix/info
>>>  setenv OPAL_MANDIR $opal_prefix/man
>>>
>>>  setenv PATH $opal_prefix/bin:$PATH
>>>  setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64
>>>
>>> Here is the error I get:
>>>
>>>  $ mpirun -np 2 hello_c
>>>  
>>> --
>>>  It looks like opal_init failed for some reason; your parallel process is
>>>  likely to abort.  There are many reasons that a parallel process can
>>>  fail during opal_init; some of which are due to configuration or
>>>  environment problems.  This failure appears to be an internal failure;
>>>  here's some additional information (which may only be relevant to an
>>>  Open MPI developer):
>>>
>>>opal_carto_base_select failed
>>>--> Returned value -13 instead of OPAL_SUCCESS
>>>  
>>> --
>>>  [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in 
>>> file runtime/orte_init.c at line 77
>>>
>>> Any ideas on what's going on?
>>>
>>> Thanks,
>>> Ethan
>> ___
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> -- 
> Jeff Squyres
> Cisco Systems
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

Can anyone get OPAL_PREFIX to work on Linux? A simple test is to see
if the following works for any mpicc/mpirun:

  $ mv  /tmp/foo
  $ set OPAL_PREFIX /tmp/foo
  $ mpicc ...
  $ mpirun ...

If you are able to get the above to run successfully, I'm interested
in your config.log file.

Thanks,
Ethan


On Thu, Dec/18/2008 11:03:25AM, Ethan Mallove wrote:
> Hello,
> 
> The below FAQ lists instructions on how to use a relocated Open MPI
> installation:
> 
>   http://www.open-mpi.org/faq/?category=building#installdirs
> 
> On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work for
> me with both MPI (hello_c) and non-MPI (hostname) programs. On Linux,
> I can only get the non-MPI case to work. Here are the environment
> variables I am setting:
> 
>   $ cat setenv_opal_prefix.csh
>   set opal_prefix = "/opt/openmpi-relocated"
> 
>   setenv OPAL_PREFIX $opal_prefix
>   setenv OPAL_BINDIR $opal_prefix/bin
>   setenv OPAL_SBINDIR$opal_prefix/sbin
>   setenv OPAL_DATAROOTDIR$opal_prefix/share
>   setenv OPAL_SYSCONFDIR $opal_prefix/etc
>   setenv OPAL_SHAREDSTATEDIR $opal_prefix/com
>   setenv OPAL_LOCALSTATEDIR  $opal_prefix/var
>   setenv OPAL_LIBDIR $opal_prefix/lib
>   setenv OPAL_INCLUDEDIR $opal_prefix/include
>   setenv OPAL_INFODIR$opal_prefix/info
>   setenv OPAL_MANDIR $opal_prefix/man
> 
>   setenv PATH $opal_prefix/bin:$PATH
>   setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64
> 
> Here is the error I get:
> 
>   $ mpirun -np 2 hello_c
>   --
>   It looks like opal_init failed for some reason; your parallel process is
>   likely to abort.  There are many reasons that a parallel process can
>   fail during opal_init; some of which are due to configuration or
>   environment problems.  This failure appears to be an internal failure;
>   here's some additional information (which may only be relevant to an
>   Open MPI developer):
> 
> opal_carto_base_select failed
> --> Returned value -13 instead of OPAL_SUCCESS
>   --
>   [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in 
> file runtime/orte_init.c at line 77
> 
> Any ideas on what's going on?
> 
> Thanks,
> Ethan

[OMPI users] Relocating an Open MPI installation using OPAL_PREFIX

[Ethan Mallove]

Hello,

The below FAQ lists instructions on how to use a relocated Open MPI
installation:

  http://www.open-mpi.org/faq/?category=building#installdirs

On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work for
me with both MPI (hello_c) and non-MPI (hostname) programs. On Linux,
I can only get the non-MPI case to work. Here are the environment
variables I am setting:

  $ cat setenv_opal_prefix.csh
  set opal_prefix = "/opt/openmpi-relocated"

  setenv OPAL_PREFIX $opal_prefix
  setenv OPAL_BINDIR $opal_prefix/bin
  setenv OPAL_SBINDIR$opal_prefix/sbin
  setenv OPAL_DATAROOTDIR$opal_prefix/share
  setenv OPAL_SYSCONFDIR $opal_prefix/etc
  setenv OPAL_SHAREDSTATEDIR $opal_prefix/com
  setenv OPAL_LOCALSTATEDIR  $opal_prefix/var
  setenv OPAL_LIBDIR $opal_prefix/lib
  setenv OPAL_INCLUDEDIR $opal_prefix/include
  setenv OPAL_INFODIR$opal_prefix/info
  setenv OPAL_MANDIR $opal_prefix/man

  setenv PATH $opal_prefix/bin:$PATH
  setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64

Here is the error I get:

  $ mpirun -np 2 hello_c
  --
  It looks like opal_init failed for some reason; your parallel process is
  likely to abort.  There are many reasons that a parallel process can
  fail during opal_init; some of which are due to configuration or
  environment problems.  This failure appears to be an internal failure;
  here's some additional information (which may only be relevant to an
  Open MPI developer):

opal_carto_base_select failed
--> Returned value -13 instead of OPAL_SUCCESS
  --
  [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
runtime/orte_init.c at line 77

Any ideas on what's going on?

Thanks,
Ethan

[OMPI users] FW: Re: [MTT users] Is the stock MPI that comes with OSX leopard broken with xgrid?

[Ethan Mallove]

Hi John,

I'm forwarding your question to the Open MPI users list.

Regards,
Ethan

On Wed, Dec/17/2008 08:35:00AM, John Fink wrote:
>Hello OpenMPI folks,
> 
>I've got a large pool of Macs running Leopard that are all on an xgrid.
>However, I can't seem to use the mpirun that comes with Leopard with the
>xgrid.  I've got my grid and password environment variables set up okay on
>my controller, all the xgrid command line commands work (displaying grid
>IDs, things like that) but mpirun only wants to run things on the local
>host.
> 
>I'm extremely new to OpenMPI and only slightly less new to Macs so there's
>probably something very obvious that I'm missing, but I'm trying what's
>detailed on this page:
>http://www.macresearch.org/runing_mpi_job_through_xgrid (the /bin/hostname
>example).  Here's my output:
> 
>as-0003-l:~ locadmin$ mpirun -n 8 /bin/hostname
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
>as-0003-l.lib.mcmaster.ca
> 
>Issuing the same command with -nolocal yields the following:
> 
>as-0003-l:~ locadmin$ mpirun --nolocal -n 8 /bin/hostname
>--
>There are no available nodes allocated to this job. This could be because
>no nodes were found or all the available nodes were already used.
> 
>Note that since the -nolocal option was given no processes can be
>launched on the local node.
>--
>[as-0003-l.lib.mcmaster.ca:82776] [0,0,0] ORTE_ERROR_LOG: Temporarily out
>of resource in file
>
> /SourceCache/openmpi/openmpi-5/openmpi/orte/mca/rmaps/base/rmaps_base_support_fns.c
>at line 168
>[as-0003-l.lib.mcmaster.ca:82776] [0,0,0] ORTE_ERROR_LOG: Temporarily out
>of resource in file
>
> /SourceCache/openmpi/openmpi-5/openmpi/orte/mca/rmaps/round_robin/rmaps_rr.c
>at line 402
>[as-0003-l.lib.mcmaster.ca:82776] [0,0,0] ORTE_ERROR_LOG: Temporarily out
>of resource in file
>
> /SourceCache/openmpi/openmpi-5/openmpi/orte/mca/rmaps/base/rmaps_base_map_job.c
>at line 210
>[as-0003-l.lib.mcmaster.ca:82776] [0,0,0] ORTE_ERROR_LOG: Temporarily out
>of resource in file
>/SourceCache/openmpi/openmpi-5/openmpi/orte/mca/rmgr/urm/rmgr_urm.c at
>line 372
>[as-0003-l.lib.mcmaster.ca:82776] mpirun: spawn failed with errno=-3
> 
>Thanks very much for any help you can provide!
> 
>jf
> 
>--
>http://libgrunt.blogspot.com -- library culture and technology.
> 
> References
> 
>Visible links
>. http://www.macresearch.org/runing_mpi_job_through_xgrid
>. http://as-0003-l.lib.mcmaster.ca/
>. http://as-0003-l.lib.mcmaster.ca/
>. http://as-0003-l.lib.mcmaster.ca/
>. file:///tmp/http:/as-0003-l.lib.mcmaster.ca:82776
>. file:///tmp/http:/as-0003-l.lib.mcmaster.ca:82776
>. file:///tmp/http:/as-0003-l.lib.mcmaster.ca:82776
>. file:///tmp/http:/as-0003-l.lib.mcmaster.ca:82776
>. file:///tmp/http:/as-0003-l.lib.mcmaster.ca:82776
>. http://libgrunt.blogspot.com/

> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] [MTT bugs] [MTT] #258: Compare non-contiguous date ranges

[Ethan Mallove]

Folks, 

I have a fix for this ticket. The syntax to get
non-contiguous date ranges is to use space-delimited quoted
tokens (see http://tinyurl.com/6e2pqa), e.g.,

  "2007-10-29 - 2007-11-30" "2008-11-07 - 2008-11-14" "by month"

Trying to view year-old results alongside week-old results in the
current Reporter can result in 10+ minutes of database thrashing. With
this fix, it's around 30 seconds. If I don't get any bug reports about
the above ~emallove Reporter within 48 hours, I'll push it out to the
main www.open-mpi.org site.

Thanks,
Ethan


On Wed, Aug/08/2007 07:22:14PM, MTT wrote:
> #258: Compare non-contiguous date ranges
> -+--
>  Reporter:  jjhursey |   Owner:
>  Type:  defect   |  Status:  new   
>  Priority:  major|   Milestone:  Future
> Component:  Server side  | Version:  trunk 
>  Keywords:   |  
> -+--
>  It is possible to get results from two types of runs (say threaded and
>  non-threaded) by regular expression arguments to some existing reporter
>  fields (i,e., configure arguments).
> 
>  However it is currently not possible to compare two distinct date ranges.
>  We should extend the reporter to effectively display such results.
> 
>  We could extend the date specification to allow for a comma separated list
>  of ranges:
>  {{{
>  To compare results from July 20, 21, and 22:
>2007-07-20 - 2007-07-21,2007-07-21 - 2007-07-22,2007-07-22 - 2007-07-23
>  }}}
> 
>  Then the reporter could make distinct summary tables for each range. Or
>  something like that.
> 
> -- 
> Ticket URL: 
> MTT 
> Issue tracking for the MPI Testing Tool.
> 

Re: [MTT users] Fwd: [Alert] Found server-side submit error messages

[Ethan Mallove]

On Thu, Oct/30/2008 01:46:58PM, Josh Hursey wrote:
> This is probably me. I haven't had a chance to do anything about it yet. 
> Hopefully tomorrow.
>
> I'm running the release branch (I believe), does this option exist for the 
> release branch yet?

It does not.

-Ethan

>
> -- Josh
>
> On Oct 30, 2008, at 11:36 AM, Ethan Mallove wrote:
>
>> On Wed, Oct/29/2008 09:15:37AM, Ethan Mallove wrote:
>>> On Tue, Oct/28/2008 06:17:12PM, Jeff Squyres wrote:
>>>> Should we set a default value of report_after_n_results to, say, 50 or 
>>>> 100?
>>>>
>>>
>>> We should.
>>>
>>> -Ethan
>>>
>>>
>>>> On Oct 28, 2008, at 6:15 PM, Jeff Squyres wrote:
>>>>
>>>>> That host is in one of IU's clusters (odin).
>>>>>
>>>>> Tim/Josh -- this is you guys...
>>
>> Got another submit.php failure alert last night from IU. If
>> the IU tests are running on the MTT trunk, an "svn up" on it
>> should eliminate the issue. (report_after_n_results now
>> defaults to 100 - see r1239.)
>>
>> -Ethan
>>
>>
>>>>>
>>>>>
>>>>> On Oct 28, 2008, at 3:45 PM, Ethan Mallove wrote:
>>>>>
>>>>>> Folks,
>>>>>>
>>>>>> I got an alert from the http-log-checker.pl script. Somebody appears 
>>>>>> to
>>>>>> have
>>>>>> lost some MTT results. (Possibly due to an oversized database 
>>>>>> submission
>>>>>> to
>>>>>> submit/index.php?) There's an open ticket for this (see
>>>>>> https://svn.open-mpi.org/trac/mtt/ticket/375).  Currently there exists 
>>>>>> a
>>>>>> simple
>>>>>> workaround for this problem. Put the below line in the problematic 
>>>>>> "Test
>>>>>> run"
>>>>>> section(s). This will prevent oversided submissions by directing MTT 
>>>>>> to
>>>>>> submit
>>>>>> the results in batches of 50 results instead of an entire section at a
>>>>>> time,
>>>>>> which can reach 400+ for an Intel test run section.
>>>>>>
>>>>>>  report_after_n_results = 50
>>>>>>
>>>>>> It's hard to know whose errors are in the HTTP error log with only the 
>>>>>> IP
>>>>>> address. If you want to verify they are or are not yours, visit a 
>>>>>> bogus
>>>>>> URL off
>>>>>> open-mpi.org, e.g., www.open-mpi.org/what-is-foobars-ip-address, and 
>>>>>> ping
>>>>>> me
>>>>>> about it. This will write your IP address to the log file, and then 
>>>>>> this
>>>>>> can be
>>>>>> matched with the IP addr against the submit.php errors.
>>>>>>
>>>>>> -Ethan
>>>>>>
>>>>>>
>>>>>> - Forwarded message from Ethan Mallove <emall...@osl.iu.edu> -
>>>>>>
>>>>>> From: Ethan Mallove <emall...@osl.iu.edu>
>>>>>> Date: Tue, 28 Oct 2008 08:00:41 -0400
>>>>>> To: ethan.mall...@sun.com, http-log-checker.pl-no-re...@open-mpi.org
>>>>>> Subject: [Alert] Found server-side submit error messages
>>>>>> Original-recipient: rfc822;ethan.mall...@sun.com
>>>>>>
>>>>>> This email was automatically sent by http-log-checker.pl. You have
>>>>>> received
>>>>>> it because some error messages were found in the HTTP(S) logs that
>>>>>> might indicate some MTT results were not successfully submitted by the
>>>>>> server-side PHP submit script (even if the MTT client has not
>>>>>> indicated a submission error).
>>>>>>
>>>>>> ###
>>>>>> #
>>>>>> # The below log messages matched "gz.*submit/index.php" in
>>>>>> # /var/log/httpd/www.open-mpi.org/ssl_error_log
>>>>>> #
>>>>>> ###
>>>>>>
>>>>>> [client 129.79.240.114] PHP Warning:  gzeof(): supplied argument is 
>>>>>> not a
>>>>>> valid stream resource in
>>>>>> /nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php 
>>>>>> on
>>>>>> line 1923
>>>>>> [client 129.79.240.114] PHP Warning:  gzgets(): supplied argument is 
>>>>>> not
>>>>>> a valid stream resource in
>>>>>> /nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php 
>>>>>> on
>>>>>> line 1924
>>>>>> ...
>>>>>> 
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> - End forwarded message -
>>>>>> ___
>>>>>> mtt-users mailing list
>>>>>> mtt-us...@open-mpi.org
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>
>>>>>
>>>>> -- 
>>>>> Jeff Squyres
>>>>> Cisco Systems
>>>>>
>>>>> ___
>>>>> mtt-users mailing list
>>>>> mtt-us...@open-mpi.org
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>
>>>>
>>>> -- 
>>>> Jeff Squyres
>>>> Cisco Systems
>>>>
>>>> ___
>>>> mtt-users mailing list
>>>> mtt-us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>> ___
>>> mtt-users mailing list
>>> mtt-us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>

Re: [MTT users] Fwd: [Alert] Found server-side submit error messages

[Ethan Mallove]

On Wed, Oct/29/2008 09:15:37AM, Ethan Mallove wrote:
> On Tue, Oct/28/2008 06:17:12PM, Jeff Squyres wrote:
> > Should we set a default value of report_after_n_results to, say, 50 or 100?
> >
> 
> We should.
> 
> -Ethan
> 
> 
> > On Oct 28, 2008, at 6:15 PM, Jeff Squyres wrote:
> >
> >> That host is in one of IU's clusters (odin).
> >>
> >> Tim/Josh -- this is you guys...

Got another submit.php failure alert last night from IU. If
the IU tests are running on the MTT trunk, an "svn up" on it
should eliminate the issue. (report_after_n_results now
defaults to 100 - see r1239.)

-Ethan


> >>
> >>
> >> On Oct 28, 2008, at 3:45 PM, Ethan Mallove wrote:
> >>
> >>> Folks,
> >>>
> >>> I got an alert from the http-log-checker.pl script. Somebody appears to 
> >>> have
> >>> lost some MTT results. (Possibly due to an oversized database submission 
> >>> to
> >>> submit/index.php?) There's an open ticket for this (see
> >>> https://svn.open-mpi.org/trac/mtt/ticket/375).  Currently there exists a 
> >>> simple
> >>> workaround for this problem. Put the below line in the problematic "Test 
> >>> run"
> >>> section(s). This will prevent oversided submissions by directing MTT to 
> >>> submit
> >>> the results in batches of 50 results instead of an entire section at a 
> >>> time,
> >>> which can reach 400+ for an Intel test run section.
> >>>
> >>>   report_after_n_results = 50
> >>>
> >>> It's hard to know whose errors are in the HTTP error log with only the IP
> >>> address. If you want to verify they are or are not yours, visit a bogus 
> >>> URL off
> >>> open-mpi.org, e.g., www.open-mpi.org/what-is-foobars-ip-address, and ping 
> >>> me
> >>> about it. This will write your IP address to the log file, and then this 
> >>> can be
> >>> matched with the IP addr against the submit.php errors.
> >>>
> >>> -Ethan
> >>>
> >>>
> >>> - Forwarded message from Ethan Mallove <emall...@osl.iu.edu> -
> >>>
> >>> From: Ethan Mallove <emall...@osl.iu.edu>
> >>> Date: Tue, 28 Oct 2008 08:00:41 -0400
> >>> To: ethan.mall...@sun.com, http-log-checker.pl-no-re...@open-mpi.org
> >>> Subject: [Alert] Found server-side submit error messages
> >>> Original-recipient: rfc822;ethan.mall...@sun.com
> >>>
> >>> This email was automatically sent by http-log-checker.pl. You have 
> >>> received
> >>> it because some error messages were found in the HTTP(S) logs that
> >>> might indicate some MTT results were not successfully submitted by the
> >>> server-side PHP submit script (even if the MTT client has not
> >>> indicated a submission error).
> >>>
> >>> ###
> >>> #
> >>> # The below log messages matched "gz.*submit/index.php" in
> >>> # /var/log/httpd/www.open-mpi.org/ssl_error_log
> >>> #
> >>> ###
> >>>
> >>> [client 129.79.240.114] PHP Warning:  gzeof(): supplied argument is not a 
> >>> valid stream resource in 
> >>> /nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php on 
> >>> line 1923
> >>> [client 129.79.240.114] PHP Warning:  gzgets(): supplied argument is not 
> >>> a valid stream resource in 
> >>> /nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php on 
> >>> line 1924
> >>> ...
> >>> 
> >>>
> >>>
> >>>
> >>>
> >>> - End forwarded message -
> >>> ___
> >>> mtt-users mailing list
> >>> mtt-us...@open-mpi.org
> >>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>
> >>
> >> -- 
> >> Jeff Squyres
> >> Cisco Systems
> >>
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >
> >
> > -- 
> > Jeff Squyres
> > Cisco Systems
> >
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] Fwd: [Alert] Found server-side submit error messages

[Ethan Mallove]

Folks,

I got an alert from the http-log-checker.pl script. Somebody appears to have
lost some MTT results. (Possibly due to an oversized database submission to
submit/index.php?) There's an open ticket for this (see
https://svn.open-mpi.org/trac/mtt/ticket/375).  Currently there exists a simple
workaround for this problem. Put the below line in the problematic "Test run"
section(s). This will prevent oversided submissions by directing MTT to submit
the results in batches of 50 results instead of an entire section at a time,
which can reach 400+ for an Intel test run section.

report_after_n_results = 50

It's hard to know whose errors are in the HTTP error log with only the IP
address. If you want to verify they are or are not yours, visit a bogus URL off
open-mpi.org, e.g., www.open-mpi.org/what-is-foobars-ip-address, and ping me
about it. This will write your IP address to the log file, and then this can be
matched with the IP addr against the submit.php errors.

-Ethan


- Forwarded message from Ethan Mallove <emall...@osl.iu.edu> -----

From: Ethan Mallove <emall...@osl.iu.edu>
List-Post: mtt-users@lists.open-mpi.org
Date: Tue, 28 Oct 2008 08:00:41 -0400
To: ethan.mall...@sun.com, http-log-checker.pl-no-re...@open-mpi.org
Subject: [Alert] Found server-side submit error messages
Original-recipient: rfc822;ethan.mall...@sun.com

This email was automatically sent by http-log-checker.pl. You have received
it because some error messages were found in the HTTP(S) logs that
might indicate some MTT results were not successfully submitted by the
server-side PHP submit script (even if the MTT client has not
indicated a submission error).

###
#
# The below log messages matched "gz.*submit/index.php" in 
# /var/log/httpd/www.open-mpi.org/ssl_error_log
#
###

[client 129.79.240.114] PHP Warning:  gzeof(): supplied argument is not a valid 
stream resource in 
/nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php on line 
1923
[client 129.79.240.114] PHP Warning:  gzgets(): supplied argument is not a 
valid stream resource in 
/nfs/rontok/xraid/data/osl/www/www.open-mpi.org/mtt/submit/index.php on line 
1924
...





- End forwarded message -

Re: [OMPI users] OpenMPI 1.2.8 on Solaris: configure problems

[Ethan Mallove]

On Fri, Oct/17/2008 05:53:07PM, Paul Kapinos wrote:
> Hi guys,
>
> did you test OpenMPI 1.2.8 on Solaris at all?!

We built 1.2.8 on Solaris successfully a few days ago:

  http://www.open-mpi.org/mtt/index.php?do_redir=869

But due to hardware/software/man-hour resource limitations,
there are often combinations of configure options, mpirun
options, etc. that end up going untested. E.g., I see you're
using some configure options we haven't tried:

 * --enable-ltdl-convenience 
 * --no-create 
 * --no-recursion
 * GCC on Solaris 


> We tried to compile OpenMPI 1.2.8 on Solaris on Sparc and on Opteron for 
> both GCC and SUN Studio compiler, in 32bit and 64bit versions, at all 
> 2*2*2=8 versions, in the very same maneer we have installed 1.2.5 and 1.2.6 
> versions.
>
>
> The configuring processes runs through, but if "gmake all" called, it seems 
> to be so, that the configure stage restarts or being resumed:
>
> ..
> orte/mca/smr/bproc/Makefile.am:47: Libtool library used but `LIBTOOL' is 
> undefined
> orte/mca/smr/bproc/Makefile.am:47:   The usual way to define `LIBTOOL' is 
> to add `AC_PROG_LIBTOOL'
> orte/mca/smr/bproc/Makefile.am:47:   to `configure.ac' and run `aclocal' 
> and `autoconf' again.
> orte/mca/smr/bproc/Makefile.am:47:   If `AC_PROG_LIBTOOL' is in 
> `configure.ac', make sure
> orte/mca/smr/bproc/Makefile.am:47:   its definition is in aclocal's search 
> path.
> test/support/Makefile.am:29: library used but `RANLIB' is undefined
> test/support/Makefile.am:29:   The usual way to define `RANLIB' is to add 
> `AC_PROG_RANLIB'
> test/support/Makefile.am:29:   to `configure.ac' and run `autoconf' again.
>
> . and breaks.

I'm confused why aclocal (or are these automake errors?) is
getting invoked in "gmake all". Did you try running
"aclocal" and "autoconf" in the top-level directory? (You
shouldn't have to do that, but it might resolve this
problem.) Make sure "ranlib" is in your PATH, mine's at
/usr/ccs/bin/ranlib.

(Also, we don't have a sys/bproc.h file on our lab machine,
so the above might be an untested scenario.)

>
> If "gmake all" again we also see error messages like:
>
> *** Fortran 77 compiler
> checking for gfortran... gfortran
> checking whether we are using the GNU Fortran 77 compiler... yes
> checking whether gfortran accepts -g... yes
> checking if Fortran 77 compiler works... yes
> checking gfortran external symbol convention... ./configure: line 26340: 
> ./conftest.o: Permission denied
> ./configure: line 26342: ./conftest.o: Permission denied
> ./configure: line 26344: ./conftest.o: Permission denied
> ./configure: line 26346: ./conftest.o: Permission denied
> ./configure: line 26348: ./conftest.o: Permission denied
> configure: error: Could not determine Fortran naming convention.
>

We didn't test 1.2.8 with GCC/Solaris. Let me see if we can
reproduce this, and get back to you.

>
> Considered the configure script we see on these lines in ./configire:
>
> if $NM conftest.o | grep foo_bar__ >/dev/null 2>&1 ; then
>   ompi_cv_f77_external_symbol="double underscore"
> elif $NM conftest.o | grep foo_bar_ >/dev/null 2>&1 ; then
> ompi_cv_f77_external_symbol="single underscore"
> elif $NM conftest.o | grep FOO_bar >/dev/null 2>&1 ; then
> ompi_cv_f77_external_symbol="mixed case"
> elif $NM conftest.o | grep foo_bar >/dev/null 2>&1 ; then
> ompi_cv_f77_external_symbol="no underscore"
> elif $NM conftest.o | grep FOO_BAR >/dev/null 2>&1 ; then
> ompi_cv_f77_external_symbol="upper case"
> else
> $NM conftest.o >conftest.out 2>&1
>
> and searching through ./configire says us, that $NM is never set 
> (neither in ./configure nor in our environment)
>

Is "nm" in your path? I have this in my config.log file:

  NM='/usr/ccs/bin/nm -p'

Thanks,
Ethan


>
> So, we think that somewhat is not OK with ./configure script. Attend to the 
> fact, that we were able to install 1.2.5 and 1.2.6 some time ago on same 
> boxes without problems.
>
> Or maybe we do somewhat wrong?
>

> best regards,
> Paul Kapinos
> HPC Group RZ RWTH Aachen
>
> P.S. Folks, does somebody compiled OpenMPI 1.2.8 on someone Solaris 
> sucessfully?
>
>
> This file contains any messages produced by compilers while
> running configure, to aid debugging if configure makes a mistake.
> 
> It was created by Open MPI configure 1.2.8, which was
> generated by GNU Autoconf 2.61.  Invocation command line was
> 
>   $ ./configure --with-devel-headers CFLAGS=-O2 -m64 CXXFLAGS=-O2 -m64 
> FFLAGS=-O2 -m64 FCFLAGS=-O2 -m64 LDFLAGS=-O2 -m64 
> --prefix=/rwthfs/rz/SW/MPI/openmpi-1.2.8/solx8664/gcc CC=gcc CXX=g++ 
> FC=gfortran --enable-ltdl-convenience --no-create --no-recursion
> 
> ## - ##
> ## Platform. ##
> ## - ##
> 
> hostname = sunoc63.rz.RWTH-Aachen.DE
> uname -m = i86pc
> uname -r = 5.10
> uname -s = SunOS
> uname -v = Generic_137112-06
> 
> /usr/bin/uname -p = i386
> /bin/uname -X = System = SunOS
> Node = 

Re: [OMPI users] Problem building OpenMPi with SunStudio compilers

[Ethan Mallove]

On Mon, Oct/06/2008 12:24:48PM, Ray Muno wrote:
> Ethan Mallove wrote:
> 
> >> Now I get farther along but the build fails at (small excerpt)
> >>
> >> mutex.c:(.text+0x30): multiple definition of `opal_atomic_cmpset_32'
> >> asm/.libs/libasm.a(asm.o):asm.c:(.text+0x30): first defined here
> >> threads/.libs/mutex.o: In function `opal_atomic_cmpset_64':
> >> mutex.c:(.text+0x50): multiple definition of `opal_atomic_cmpset_64'
> >> asm/.libs/libasm.a(asm.o):asm.c:(.text+0x50): first defined here
> >> make[2]: *** [libopen-pal.la] Error 1
> >> make[2]: Leaving directory 
> >> `/home/muno/OpenMPI/SunStudio/openmpi-1.2.7/opal'
> >> make[1]: *** [all-recursive] Error 1
> >> make[1]: Leaving directory 
> >> `/home/muno/OpenMPI/SunStudio/openmpi-1.2.7/opal'
> >> make: *** [all-recursive] Error 1
> >>
> >> I based the configure on what was found in the FAQ here.
> >>
> >> http://www.open-mpi.org/faq/?category=building#build-sun-compilers
> >>
> >> Perhaps this is much more specific to our platform/OS.
> >>
> >> The environment is AMD Opteron, Barcelona running Centos 5
> >> (Rocks 5.03) with SunStudio 12 compilers.
> >>
> > 
> > Unfortunately I haven't seen the above issue, so I don't
> > have a workaround to propose. There are some issues that
> > have been fixed with GCC-style inline assembly in the latest
> > Sun Studio Express build. Could you try it out?
> > 
> >   http://developers.sun.com/sunstudio/downloads/express/index.jsp
> > 
> > -Ethan
> > 
> > 
> 
> Looks like it dies at the exact same spot. I have the C++
> failure as well (supplied ld does not work).
> 

Can you send your full config.log. This will help in an
attempt to reproduce your issue.

-Ethan


> -- 
> 
>  Ray Muno   http://www.aem.umn.edu/people/staff/muno
>  University of Minnesota   e-mail:   m...@aem.umn.edu
>  Aerospace Engineering and MechanicsPhone: (612) 625-9531
>  110 Union St. S.E.   FAX: (612) 626-1558
>  Minneapolis, Mn 55455
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Problem building OpenMPi with SunStudio compilers

[Ethan Mallove]

On Sat, Oct/04/2008 11:21:27AM, Raymond Muno wrote:
> Raymond Muno wrote:
>> Raymond Muno wrote:
>>> We are implementing a new cluster that is InfiniBand based.  I am working 
>>> on getting OpenMPI built for our various compile environments. So far it 
>>> is working for PGI 7.2 and PathScale 3.1.  I found some workarounds for 
>>> issues with the Pathscale compilers (seg faults) in the OpenMPI FAQ.
>>>
>>> When I try to build with SunStudio, I cannot even get past the configure 
>>> stage. It dies in th estage that checks for C++.
>>
>> It looks like the problem is with SunStudio itself. Even a simple CC 
>> program fails to compile.
>>>
>>> /usr/lib64/libm.so: file not recognized: File format not recognized
> OK, I took care of the linker issue fro C++ as recommended on Suns support 
> site (replace Sun supplied ld with /usr/bin/ld)
>
> Now I get farther along but the build fails at (small excerpt)
>
> mutex.c:(.text+0x30): multiple definition of `opal_atomic_cmpset_32'
> asm/.libs/libasm.a(asm.o):asm.c:(.text+0x30): first defined here
> threads/.libs/mutex.o: In function `opal_atomic_cmpset_64':
> mutex.c:(.text+0x50): multiple definition of `opal_atomic_cmpset_64'
> asm/.libs/libasm.a(asm.o):asm.c:(.text+0x50): first defined here
> make[2]: *** [libopen-pal.la] Error 1
> make[2]: Leaving directory `/home/muno/OpenMPI/SunStudio/openmpi-1.2.7/opal'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/home/muno/OpenMPI/SunStudio/openmpi-1.2.7/opal'
> make: *** [all-recursive] Error 1
>
> I based the configure on what was found in the FAQ here.
>
> http://www.open-mpi.org/faq/?category=building#build-sun-compilers
>
> Perhaps this is much more specific to our platform/OS.
>
> The environment is AMD Opteron, Barcelona running Centos 5
> (Rocks 5.03) with SunStudio 12 compilers.
>

Unfortunately I haven't seen the above issue, so I don't
have a workaround to propose. There are some issues that
have been fixed with GCC-style inline assembly in the latest
Sun Studio Express build. Could you try it out?

  http://developers.sun.com/sunstudio/downloads/express/index.jsp

-Ethan


> Does anyone have any insight as to how to successfully
> build OpenMPI for this OS/compiler selection?  As I said
> in the first post, we have it built for Pathscale 3.1 and
> PGI 7.2. 
> -Ray Muno
> University of Minnesota, Aerospace Engineering
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [MTT users] Patch to add --local-scratch option

[Ethan Mallove]

On Thu, Sep/18/2008 05:35:13PM, Jeff Squyres wrote:
> On Sep 18, 2008, at 10:45 AM, Ethan Mallove wrote:
>
>>> Ah, yeah, ok, now I see why you wouldl call it --mpi-install-scratch, so
>>> that it matches the MTT ini section name.  Sure, that works for me.
>>
>> Since this does seem like a feature that should eventually
>> propogate to all the other phases (except for Test run),
>> what will we call the option to group all the fast phase
>> scratches?
>
> --scratch
>
> :-)
>
> Seriously, *if* we ever implement the other per-phase scratches, I think 
> having one overall --scratch and fine-grained per-phase specifications 
> fine.  I don't think we need to go overboard to have a way to say I want 
> phases X, Y, and Z to use scratch A.  Meaning that you could just use 
> --X-scratch=A --Y-scratch=A and --Z-scratch=A.

--mpi-install-scratch actually has MTT install (using
DESTDIR) into --scratch. Is that confusing? Though
--fast-scratch could also be misleading, as I could see a
user thinking that --fast-scratch will do some magical
optimization to make their NFS directory go faster. I guess
I'm done splitting hairs over --mpi-install-scratch :-)

-Ethan


>
> -- 
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Patch to add --local-scratch option

[Ethan Mallove]

On Tue, Sep/16/2008 12:07:31PM, Ethan Mallove wrote:
> Nice! A few thoughts ...
> 
> What happens if one uses --local-scratch, but leaves out the
> --scratch option? In this case, I think MTT should assume 
> --scratch is the same as --local-scratch.
> 
> Could the "local" in --local-scratch ever be misleading?
> Couldn't a user ever use a mounted filesystem that's faster
> than all their local filesystem? Should it be
> --fast-scratch?
> 
> For future work, how about --scratch taking a (CSV?) list of
> scratch directories. MTT then does a quick check for which
> is the fastest filesystem (if such a check is
> possible/feasible), and proceeds accordingly. That is, doing
> everything it possible can in a fast scratch (builds,
> writing out metadata, etc.), and installing the MPI(s) into
> the slow mounted scratch. Would this be possible?
> 
> On Mon, Sep/15/2008 06:15:43PM, Tim Mattox wrote:
> > Hello,
> > Attached is a patchfile for the mtt trunk that adds a
> > --local-scratch 
> > option to client/mtt.  You can also specify something like
> > this in your [MTT] ini section:
> > local_scratch = ("echo /tmp/`whoami`_mtt")
> > 
> > This local-scratch directory is then used for part of the --mpi-install
> > phase to speed up your run.  Specifically, the source-code of the
> > MPI is untarred there, configure is run, make all, and make check.
> > Then, when make install is invoked the MPI is installed into the
> > usual place as if you hadn't used --local-scratch.  If you don't
> > use --local-scratch, then the builds occur in the usual place that
> > they have before.
> > 
> > For the clusters at IU that seem to have slow NSF home directories,
> > this cuts the --mpi-install phase time in half.
> > 
> > The downside is that if the MPI build fails, your build directory is out
> > on some compile-node's /tmp and is harder to go debug.  But, since
> > mpi build failures are now rare, this should make for quicker turnaround
> > for the general case.
> > 
> > I think I adjusted the code properly for the vpath build case, but I've
> > never used that so haven't tested it.  Also, I adjusted the free disk space
> > check code.  Right now it only checks the free space on --scratch,
> > and won't detect if --local-scratch is full.  If people really care, I
> > could make it check both later.  But for now, if your /tmp is full
> > you probably have other problems to worry about.
> > 
> > Comments?  Can you try it out, and if I get no objections, I'd like
> > to put this into the MTT trunk this week.
> 
> I'll test your patch soon.
> 

I was able to do an MPI get/install after applying the patch.

I ran:

  $ client/mtt --local-scratch /tmp/foo --scratch /nfs/bar ...

Which resulted in the build directory ending up in:

  /tmp/foo/.../src

And the resulting installation ending up in the below
directory via the DESTDIR macro passed to "make install":

  /nfs/bar/.../src

The patch seems to do what it's supposed to do.

-Ethan


> -Ethan
> 
> 
> > -- 
> > Tim Mattox, Ph.D. - http://homepage.mac.com/tmattox/
> >  tmat...@gmail.com || timat...@open-mpi.org
> >  I'm a bright... http://www.the-brights.net/
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 

Re: [MTT users] Patch to add --local-scratch option

[Ethan Mallove]

Nice! A few thoughts ...

What happens if one uses --local-scratch, but leaves out the
--scratch option? In this case, I think MTT should assume 
--scratch is the same as --local-scratch.

Could the "local" in --local-scratch ever be misleading?
Couldn't a user ever use a mounted filesystem that's faster
than all their local filesystem? Should it be
--fast-scratch?

For future work, how about --scratch taking a (CSV?) list of
scratch directories. MTT then does a quick check for which
is the fastest filesystem (if such a check is
possible/feasible), and proceeds accordingly. That is, doing
everything it possible can in a fast scratch (builds,
writing out metadata, etc.), and installing the MPI(s) into
the slow mounted scratch. Would this be possible?

On Mon, Sep/15/2008 06:15:43PM, Tim Mattox wrote:
> Hello,
> Attached is a patchfile for the mtt trunk that adds a
> --local-scratch 
> option to client/mtt.  You can also specify something like
> this in your [MTT] ini section:
> local_scratch = ("echo /tmp/`whoami`_mtt")
> 
> This local-scratch directory is then used for part of the --mpi-install
> phase to speed up your run.  Specifically, the source-code of the
> MPI is untarred there, configure is run, make all, and make check.
> Then, when make install is invoked the MPI is installed into the
> usual place as if you hadn't used --local-scratch.  If you don't
> use --local-scratch, then the builds occur in the usual place that
> they have before.
> 
> For the clusters at IU that seem to have slow NSF home directories,
> this cuts the --mpi-install phase time in half.
> 
> The downside is that if the MPI build fails, your build directory is out
> on some compile-node's /tmp and is harder to go debug.  But, since
> mpi build failures are now rare, this should make for quicker turnaround
> for the general case.
> 
> I think I adjusted the code properly for the vpath build case, but I've
> never used that so haven't tested it.  Also, I adjusted the free disk space
> check code.  Right now it only checks the free space on --scratch,
> and won't detect if --local-scratch is full.  If people really care, I
> could make it check both later.  But for now, if your /tmp is full
> you probably have other problems to worry about.
> 
> Comments?  Can you try it out, and if I get no objections, I'd like
> to put this into the MTT trunk this week.

I'll test your patch soon.

-Ethan


> -- 
> Tim Mattox, Ph.D. - http://homepage.mac.com/tmattox/
>  tmat...@gmail.com || timat...@open-mpi.org
>  I'm a bright... http://www.the-brights.net/
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Can not find my testing results in OMPI MTT DB

[Ethan Mallove]

I see a bunch of these errors in the httpd logs today:

  PHP Fatal error:  Allowed memory size of 33554432

Could you send your INI file? One way around this issue is
to submit fewer test results at one time by breaking up your
large MTT run into multiple runs. If you're running on the
trunk, this parameter will submit results after each run.

-Ethan


On Tue, Jul/08/2008 10:22:47AM, Pavel Shamis (Pasha) wrote:
> Hi All,
> I still see that some time MTT (http://www.open-mpi.org/mtt) "lose" my 
> testing results, in the local log I see:
>
>   ### Test progress: 474 of 474 section tests complete (100%)
> Submitting to MTTDatabase...
> MTTDatabase client trying proxy:  / Default (none)
> MTTDatabase proxy successful / not 500
> MTTDatabase response is a success
> MTTDatabase client got response:
> *** WARNING: MTTDatabase client did not get a serial; phases will be
>isolated from each other in the reports
> MTTDatabase client submit complete
>
> And I can not find these results in DB.
> Is it any progress with this issue ?
>
>
> Regards.
> Pasha
>
> Ethan Mallove wrote:
>> On Wed, May/21/2008 09:53:11PM, Pavel Shamis (Pasha) wrote:
>>   
>>> Oops, in the "MTT server side problem" we discussed other issue.
>>>
>>> But anyway I did not see the problem on my server after the upgrade :)
>>> 
>>
>>
>> We took *some* steps to alleviate the PHP memory overload
>> problem (e.g., r668, and then r1119), but evidently there's
>> more work to do :-)
>>
>>   
>>> Pasha
>>>
>>> Pavel Shamis (Pasha) wrote:
>>> 
>>>> I had similar problem on my server. I upgraded the server to latest 
>>>> trunk and the problem disappear.
>>>> (see "MTT server side problem" thread).
>>>>
>>>> Pasha
>>>>
>>>> Jeff Squyres wrote:
>>>> 
>>>>> FWIW: I think we have at least one open ticket on this issue (break up 
>>>>> submits so that we don't overflow PHP and/or apache).
>>>>> 
>>
>> https://svn.open-mpi.org/trac/mtt/ticket/221
>>
>> -Ethan
>>
>>   
>>>>> On May 21, 2008, at 2:36 PM, Ethan Mallove wrote:
>>>>>
>>>>> 
>>>>>> On Wed, May/21/2008 06:46:06PM, Pavel Shamis (Pasha) wrote:
>>>>>> 
>>>>>>> I sent it directly to your email. Please check.
>>>>>>> Thanks,
>>>>>>> Pasha
>>>>>>> 
>>>>>> Got it. Thanks. It's a PHP memory overload issue.
>>>>>> (Apparently I didn't look far back enough in the httpd
>>>>>> error_logs.) See below.
>>>>>>
>>>>>> 
>>>>>>> Ethan Mallove wrote:
>>>>>>> 
>>>>>>>> On Wed, May/21/2008 05:19:44PM, Pavel Shamis (Pasha) wrote:
>>>>>>>>
>>>>>>>> 
>>>>>>>>> Jeff Squyres wrote:
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>>> Are we running into http max memory problems or http max upload size
>>>>>>>>>> problems again?
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>> I guess it is some server side issue, you need to check the
>>>>>>>>> /var/log/httpd/* log on the server.
>>>>>>>>>
>>>>>>>>> 
>>>>>>>> The only thing I found in the httpd logs
>>>>>>>> (/var/log/httpd/www.open-mpi.org/error_log*) was this PHP
>>>>>>>> warning, which I don't think would result in lost results:
>>>>>>>>
>>>>>>>> PHP Warning:  array_shift(): The argument should be an array in 
>>>>>>>> .../submit/index.php on line 1683
>>>>>>>>
>>>>>>>> I haven't received any emailed Postgres errors either. When
>>>>>>>> were these results submitted? I searched for "mellanox" over
>>>>>>>> the past four days. It seem the results aren't buried in
>>>>>>>> here, because I see no test run failures ...
>>>>>>>>
>>>>

Re: [MTT users] Can not find my testing results in OMPI MTT DB

[Ethan Mallove]

On Wed, May/21/2008 09:53:11PM, Pavel Shamis (Pasha) wrote:
> Oops, in the "MTT server side problem" we discussed other issue.
>
> But anyway I did not see the problem on my server after the upgrade :)


We took *some* steps to alleviate the PHP memory overload
problem (e.g., r668, and then r1119), but evidently there's
more work to do :-)

>
> Pasha
>
> Pavel Shamis (Pasha) wrote:
>> I had similar problem on my server. I upgraded the server to latest trunk 
>> and the problem disappear.
>> (see "MTT server side problem" thread).
>>
>> Pasha
>>
>> Jeff Squyres wrote:
>>   
>>> FWIW: I think we have at least one open ticket on this issue (break up 
>>> submits so that we don't overflow PHP and/or apache).

https://svn.open-mpi.org/trac/mtt/ticket/221

-Ethan

>>>
>>> On May 21, 2008, at 2:36 PM, Ethan Mallove wrote:
>>>
>>> 
>>>> On Wed, May/21/2008 06:46:06PM, Pavel Shamis (Pasha) wrote:
>>>>   
>>>>> I sent it directly to your email. Please check.
>>>>> Thanks,
>>>>> Pasha
>>>>> 
>>>> Got it. Thanks. It's a PHP memory overload issue.
>>>> (Apparently I didn't look far back enough in the httpd
>>>> error_logs.) See below.
>>>>
>>>>   
>>>>> Ethan Mallove wrote:
>>>>> 
>>>>>> On Wed, May/21/2008 05:19:44PM, Pavel Shamis (Pasha) wrote:
>>>>>>
>>>>>>   
>>>>>>> Jeff Squyres wrote:
>>>>>>>
>>>>>>> 
>>>>>>>> Are we running into http max memory problems or http max upload size
>>>>>>>> problems again?
>>>>>>>>
>>>>>>>>   
>>>>>>> I guess it is some server side issue, you need to check the
>>>>>>> /var/log/httpd/* log on the server.
>>>>>>>
>>>>>>> 
>>>>>> The only thing I found in the httpd logs
>>>>>> (/var/log/httpd/www.open-mpi.org/error_log*) was this PHP
>>>>>> warning, which I don't think would result in lost results:
>>>>>>
>>>>>> PHP Warning:  array_shift(): The argument should be an array in 
>>>>>> .../submit/index.php on line 1683
>>>>>>
>>>>>> I haven't received any emailed Postgres errors either. When
>>>>>> were these results submitted? I searched for "mellanox" over
>>>>>> the past four days. It seem the results aren't buried in
>>>>>> here, because I see no test run failures ...
>>>>>>
>>>>>>  http://www.open-mpi.org/mtt/index.php?do_redir=659
>>>>>>
>>>>>> I'm assuming you're running with two Reporter INI sections:
>>>>>> Textfile and MTTDatabase? Can you send some MTT client
>>>>>> --verbose/--debug output from the below runs?
>>>>>>
>>>>>> Thanks,
>>>>>> Ethan
>>>>>>
>>>>>>
>>>>>>   
>>>>>>>> On May 21, 2008, at 5:28 AM, Pavel Shamis (Pasha) wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>   
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Here is test result from my last mtt run:
>>>>>>>>> +-++--+--+--+--+
>>>>>>>>> | Phase   | Section| Pass | Fail | Time out | Skip |
>>>>>>>>> +-++--+--+--+--+
>>>>>>>>> | MPI install | ompi/gcc   | 1| 0| 0| 0|
>>>>>>>>> | MPI install | ompi/intel-9.0 | 1| 0| 0| 0|
>>>>>>>>> | Test Build  | trivial| 1| 0| 0| 0|
>>>>>>>>> | Test Build  | trivial| 1| 0| 0| 0|
>>>>>>>>> | Test Build  | intel-suite| 1| 0| 0| 0|
>>>>>>>>> | Test Build  | intel-suite| 1| 0| 0| 0|
>>>>>>>>> | Test Build  | imb| 1| 0| 0| 0|
>>>>>>>>> | Test Build  | 

Re: [MTT users] MTT server side problem

[Ethan Mallove]

On Tue, May/06/2008 06:29:33PM, Pavel Shamis (Pasha) wrote:
> 
> > I'm not sure which cron jobs you're referring to. Do you
> > mean these?
> >
> >   https://svn.open-mpi.org/trac/mtt/browser/trunk/server/php/cron
> >   
> I talked about this one: 
> https://svn.open-mpi.org/trac/mtt/wiki/ServerMaintenance

I'm guessing you would only be concerned with the below
periodic-maintenance.pl script, which just runs
ANALYZE/VACUUM queries. I think you can start that up
whenever you want (and it should optimize the Reporter).

  
https://svn.open-mpi.org/trac/mtt/browser/trunk/server/sql/cron/periodic-maintenance.pl

-Ethan


> 
> > The only thing there are the regular
> > mtt-resu...@open-mpi.org email alerts and some out-of-date
> > DB monitoring junk. You can ignore that stuff.
> >
> > Josh, are there some nightly (DB
> > pruning/cleaning/vacuuming?) cron jobs that Pasha should be
> > running?
> >
> > -Ethan
> >
> >   
> >> Thanks.
> >>
> >> Ethan Mallove wrote:
> >> 
> >>> Hi Pasha,
> >>>
> >>> I thought this issue was solved in r1119 (see below). Do you
> >>> have the latest mtt/server scripts?
> >>>
> >>>   https://svn.open-mpi.org/trac/mtt/changeset/1119/trunk/server/php/submit
> >>>
> >>> -Ethan
> >>>
> >>> On Tue, May/06/2008 03:26:43PM, Pavel Shamis (Pasha) wrote:
> >>>   
> >>>   
> >>>> About the issue:
> >>>>  
> >>>> 1. On client side I see ""*** WARNING: MTTDatabase client did not get a 
> >>>> serial"
> >>>> As result of the error some of MTT results is not visible via the web 
> >>>> reporter
> >>>>  
> >>>> 2. On server side I found follow error message:
> >>>> [client 10.4.3.214] PHP Fatal error:  Allowed memory size of 33554432 
> >>>> bytes exhausted (tried to allocate 23592960
> >>>> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) : eval()'d 
> >>>> code on line 77515
> >>>> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting down
> >>>> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled (wrapper: 
> >>>> /usr/sbin/suexec)
> >>>> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret for digest 
> >>>> authentication ...
> >>>> [Mon May 05 19:30:54 2008] [notice] Digest: done
> >>>> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP LDAP SDK
> >>>> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support unavailable
> >>>>  
> >>>> My memory limit in php.ini file was set on 256MB !
> >>>>
> >>>> Any ideas ?
> >>>>
> >>>> Thanks.
> >>>>
> >>>>
> >>>> -- 
> >>>> Pavel Shamis (Pasha)
> >>>> Mellanox Technologies
> >>>>
> >>>> ___
> >>>> mtt-users mailing list
> >>>> mtt-us...@open-mpi.org
> >>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>>> 
> >>>> 
> >>>   
> >>>   
> >> -- 
> >> Pavel Shamis (Pasha)
> >> Mellanox Technologies
> >>
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >> 
> >
> >   
> 
> 
> -- 
> Pavel Shamis (Pasha)
> Mellanox Technologies
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] MTT server side problem

[Ethan Mallove]

On Tue, May/06/2008 05:34:46PM, Pavel Shamis (Pasha) wrote:
> Hi Ethan,
> I don't run the latest version. Tomorrow I will try to update the server 
> side. I will keep you updated.
> 
> Other question.
> On the server side after the server bring up we forgot to update crontab 
> to run the mtt cron. Is it ok to start the cron jobs now ?
> I afraid the the late cron start up will cause problem to DB.

I'm not sure which cron jobs you're referring to. Do you
mean these?

  https://svn.open-mpi.org/trac/mtt/browser/trunk/server/php/cron

The only thing there are the regular
mtt-resu...@open-mpi.org email alerts and some out-of-date
DB monitoring junk. You can ignore that stuff.

Josh, are there some nightly (DB
pruning/cleaning/vacuuming?) cron jobs that Pasha should be
running?

-Ethan

> 
> Thanks.
> 
> Ethan Mallove wrote:
> > Hi Pasha,
> >
> > I thought this issue was solved in r1119 (see below). Do you
> > have the latest mtt/server scripts?
> >
> >   https://svn.open-mpi.org/trac/mtt/changeset/1119/trunk/server/php/submit
> >
> > -Ethan
> >
> > On Tue, May/06/2008 03:26:43PM, Pavel Shamis (Pasha) wrote:
> >   
> >> About the issue:
> >>  
> >> 1. On client side I see ""*** WARNING: MTTDatabase client did not get a 
> >> serial"
> >> As result of the error some of MTT results is not visible via the web 
> >> reporter
> >>  
> >> 2. On server side I found follow error message:
> >> [client 10.4.3.214] PHP Fatal error:  Allowed memory size of 33554432 
> >> bytes exhausted (tried to allocate 23592960
> >> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) : eval()'d 
> >> code on line 77515
> >> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting down
> >> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled (wrapper: 
> >> /usr/sbin/suexec)
> >> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret for digest 
> >> authentication ...
> >> [Mon May 05 19:30:54 2008] [notice] Digest: done
> >> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP LDAP SDK
> >> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support unavailable
> >>  
> >> My memory limit in php.ini file was set on 256MB !
> >>
> >> Any ideas ?
> >>
> >> Thanks.
> >>
> >>
> >> -- 
> >> Pavel Shamis (Pasha)
> >> Mellanox Technologies
> >>
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >> 
> >
> >   
> 
> 
> -- 
> Pavel Shamis (Pasha)
> Mellanox Technologies
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] LD_LIBRARY_PATH setting in Run.pm masks a wrapper compiler issue

[Ethan Mallove]

Line 386 of MTT/Test/Run.pm sets LD_LIBRARY_PATH. This
appears to have masked a problem in my wrapper compiler
(missing -R/foo/lib flags).

if ($mpi_install->{libdir}) {
if (exists($ENV{LD_LIBRARY_PATH})) {
$ENV{LD_LIBRARY_PATH} = "$mpi_install->{libdir}:" . 
$ENV{LD_LIBRARY_PATH};
} else {
$ENV{LD_LIBRARY_PATH} = $mpi_install->{libdir};
}
}

Is the rationale for setting LD_LIBRARY_PATH for the
mpirun's the same as the rationale described below (from
GNU_Install.pm)?

my %ENV_SAVE = %ENV;
$ENV{TMPDIR} = "$config->{installdir}/tmp";
mkdir($ENV{TMPDIR}, 0777);
# The intent here is just to ensure that the LD_LIBRARY_PATH
# in the environment does not point to shared libraries
# outside of MTT's scope that would interfere with "make
# check" (e.g., another libmpi.so outside of MTT).  Just
# prepend our own $libdir to LD_LIBRARY_PATH and hope that
# that's Good Enough.  :-)
if (exists($ENV{LD_LIBRARY_PATH})) {
$ENV{LD_LIBRARY_PATH} = "$config->{libdir}:$ENV{LD_LIBRARY_PATH}";
} else {
$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
}

If so, could we make MTT a little smarter. E.g., actually
check for a libmpi.so (and/or other possibly conflicting
DSOs), and if there's no conflict then skip the
LD_LIBRARY_PATH setting. (This way a wrapper compiler issue
is not masked.)

-Ethan

Re: [MTT users] Introduce myself.

[Ethan Mallove]

Check this page out to get started using MTT:

  https://svn.open-mpi.org/trac/mtt/wiki/MTTOverview

-Ethan


On Sat, Apr/05/2008 10:02:49PM, Davi Vercillo C. Garcia wrote:
> Hi all,
> 
> My name is Davi Vercillo and I'm from Brazil. I'm starting right now
> to study MPI and I'll use the implementation of OpenMPI for this. I
> would like to know how I can use the MTT.
> 
> PS: Sorry about my english...
> 
> -- 
> Davi Vercillo Carneiro Garcia
> 
> Universidade Federal do Rio de Janeiro
> Departamento de Ci?ncia da Computa??o
> DCC-IM/UFRJ - http://www.dcc.ufrj.br
> 
> "Good things come to those who... wait." - Debian Project
> 
> "A computer is like air conditioning: it becomes useless when you open
> windows." - Linus Torvalds
> 
> "H? duas coisas infinitas, o universo e a burrice humana. E eu estou
> em d?vida quanto o primeiro." - Albert Einstein
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Reporter problems

[Ethan Mallove]

I don't remember a "past 24 hour" summary taking "24
seconds". Are we seeing a slow down due to an accumulation
of results? I thought the week-long table partitions would
prevent this type of effect?

-Ethan


On Wed, Jan/30/2008 11:00:46AM, Josh Hursey wrote:
> This maintenance is complete. The reporter should be operating as  
> normal.
> 
> There are a few other maintenance items, but I am pushing them to the  
> weekend since it will result in a bit of a slowdown again.
> 
> Thanks for your patience.
> 
> Cheers,
> Josh
> 
> On Jan 29, 2008, at 9:47 AM, Josh Hursey wrote:
> 
> > The reporter should be responding much better now. I tweaked the
> > maintenance scripts so they no longer push nearly as hard on the
> > database. They are still running, but the query you specified seems to
> > run in approx. 15-20 sec. with the current load.
> >
> > -- Josh
> >
> > On Jan 29, 2008, at 8:38 AM, Josh Hursey wrote:
> >
> >> For the next 24 - 48 hours this is to be expected. Sorry :(
> >>
> >> I started some maintenance work last night, and it is taking a bit
> >> longer than I expected (due to integrity constraint checking most
> >> likely). The maintenance scripts are pushing fairly hard on the
> >> database, so I would expect some slowdown with the reporter (and  
> >> maybe
> >> client submits).
> >>
> >> If this becomes a substantial problem for anyone please let me know,
> >> and I may be able to shift this work to the weekend. In the mean time
> >> I'll see if I can reduce the load a bit.
> >>
> >> -- Josh
> >>
> >>
> >> On Jan 29, 2008, at 7:44 AM, Tim Prins wrote:
> >>
> >>> Hi,
> >>>
> >>> Using the reporter this morning it is being awfully slow, as in it  
> >>> is
> >>> taking about 3 minutes to do a top level summary search for:
> >>> Date: past 24 hours
> >>> Org: IU
> >>> Platform name: IU_Odin
> >>>
> >>> I don't know whether this is a known problem or not. I seem to  
> >>> recall
> >>> that after the last database upgrade such a search was taking just a
> >>> few
> >>> seconds.
> >>>
> >>> Thanks,
> >>>
> >>> Tim
> >>> ___
> >>> mtt-users mailing list
> >>> mtt-us...@open-mpi.org
> >>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] questions on MTT setup instructions.

[Ethan Mallove]

On Wed, Jan/23/2008 03:36:26PM, Jennis Pruett wrote:
> 
>To followup on this, I think I got it right by just using:
> 
>perl Makefile.PL LIB=/usr/projects/hpctools/jennyp/mtt
>PREFIX=/usr/projects/hpctools/jennyp/mtt
>setenv PERLIB /usr/projects/hpctools/jennyp/mtt
> 
>I then copied ompi-core-template.ini  (made no changes to it).  But:
> 
>ffe2:/<2>hpctools/jennyp/mtt [205] > ./client -d --force -f
>ompi-core-template.ini
>Unable to find MTT support libraries at ./client line 78.
>BEGIN failed--compilation aborted at ./client line 81.
>ffe2:/<2>hpctools/jennyp/mtt [206] > ./client -help
>Unable to find MTT support libraries at ./client line 78.
>BEGIN failed--compilation aborted at ./client line 81.
> 
> 
>I can't help but feel that I'm doing something wrong.
>Help?  Do I need to read another document?


You may need to do this:

  $ setenv PERLIB /usr/projects/hpctools/jennyp/mtt/lib

-Ethan

> 
>original issue-
>I'm following the instructions on the MTT setup page.
> 
>Would someone clarify the  parameters below.
>PREFIX is logically (to me) "where I am putting MTT" , yes?
> 
>But LIB is always confusing. Is this some MPI lib directory,
>a perl library or the same as  above - where I am putting MTT.
> 
>And if so, how do I find where those happen
>to reside?
> 
>Pointing PERLLIB means setting the env var, right?
> 
>New to this, thanks.,
>Jenny
> 
>---
>Jennis (Jenny) Pruett
>Los Alamos National Laboratory
>505-667-1955
> 

> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] hostlist enhancement

[Ethan Mallove]

Works for me.

-Ethan

On Thu, Jan/10/2008 08:48:20AM, Jeff Squyres wrote:
> Mellanox told me that the MTT () funclet is returning a 
> comma-delimited list of hosts (and _hosts()).  That is fine for 
> Open MPI, but it is not for MVAPICH -- MVAPICH requires a space-delimited 
> list of hosts for their mpirun.
>
> Here's a patch that introduces an optional parameter to () and 
> _hosts().  The optional parameter is a delimiter for the hostlist. 
>  So if you call:
>
>   _hosts()
>
> you'll get the same comma-delimited list that is returned today.  But if 
> you call
>
>   _hosts(" ")
>
> you should get a space-delimited list.
>
> Can Mellanox try this patch and see if it works for them?  If so, I'll 
> commit it to the MTT trunk.
>
> -- 
> Jeff Squyres
> Cisco Systems


>

> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] mtt reports arrive without subject.

[Ethan Mallove]

Hi Pasha,

I confess that I don't use the Email reporter. I think the
email_subject is out-of-date. Try this:

  email_subject = MPI test results: _section()

I'm unsure about the empty email body, but have an idea.
What does this output for you?

  $ which Mail mailx mail rmail mutt elm

I'm guessing the problem is that your mail agent has a
problem with doing this:

  $ cat foo.txt |  -s "MTT Mail Test" pa...@mellanox.co.il
  $ cat foo.txt
  Subject:
  start_timestamp: blah blah
  ...

The body of the mail message *should* be something like this:

  Subject: MPI test results: mpi install: mpi-developer
  start_timestamp: Thu Jan  3 16:35:53 2008
  bitness: 4
  compiler_name: sun
  compiler_version: 5.9 2007/07/12;
  configure_arguments: blah blah ...
  duration: 0 seconds
  endian: 0
  ENVIRONMENT_BEGIN
  prepend_path PATH /ws/ompi-tools/SUNWspro/SOS12-ML/bin
  ENVIRONMENT_END
  exit_signal: -1
  exit_value: 0
  merge_stdout_stderr: 1
  mpi_install_section_name: mpi-developer
  mpi_name: ompi-nightly-v1.0
  mpi_version: 1.0.3a1r11129
  os_version: s10x_u4wos_12
  phase: MPI Install
  result_message: blah blah ...
  start_timestamp: Thu Jan  3 16:37:16 2008
  test_result: 0
  vpath_mode: 0

Regards,
Ethan



On Thu, Jan/10/2008 02:55:10PM, Pavel Shamis (Pasha) wrote:
> Hi,
> I'm running MTT trunk (r1126) and I'm getting mail reports without any 
> subject.
> 
> In ini file I have:
> [Reporter: send email]
> module = Email
> email_to = pa...@mellanox.co.il
> email_subject = MPI test results: $phase / $sectiona
> 
> In mail body I got:
> 
> Subject:
> start_timestamp: Thu Jan 10 12:25:56 2008
> 
> Thanks.
> 
> -- 
> Pavel Shamis (Pasha)
> Mellanox Technologies
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] mtt multiple mpi install and non open mpi instal issues

[Ethan Mallove]

Thanks Oleg! It looks like Jeff already applied the
multilib.patch. The other one looks good to me.

-Ethan


On Mon, Jan/07/2008 07:23:55PM, Oleg Kats wrote:
>Hello All
> 
>Here is list of issues that we figure out during ourMTT usage:
> 
> 
> 
>1.   Installation of multiple MPIs failed.
> 
>  MTT successfully installed the first mpi installation but the all
>other installations were installed under ~user directory. For solution
>please see mttlib.patch
> 
>2.   Build of MPIs other than Open-mpi failed.
> 
>  PrepareForInstall function in MTT/MPI/Get/Download.pm client was
>stay in install/src dir and not in install/src/.
> 
>  please see mttlib_download.patch
> 
> 
> 
> 



> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] SVN URLs

[Ethan Mallove]

On Thu, Dec/20/2007 11:51:30AM, Jeff Squyres wrote:
> On Dec 20, 2007, at 11:38 AM, Ethan Mallove wrote:
> 
> >> FYI: per my post to the OMPI devel list, I discovered the
> >> "svnsync"  command yesterday.  svnsync allows you to
> >> mirror an entire SVN  repository in a read-only fashion; I
> >> mirrored the ompi-tests  repository locally at Cisco.  A
> >> one-line cron job keeps my ompi-tests SVN repo mirror up
> >> to date every night.
> >
> > Can you post that crontab?
> 
> It's just
> 
> 0 19 * * * . $HOME/.bashrc ; svnsync sync 
> file:///home/mpiteam/mirrors-svnroots/ompi-test
> 
> (repeated for the other SVN repo's I'm mirroring)
> 
> See the SVN book docs for svnsync -- they discuss how to setup a local  
> repo for mirroring with svnsync (there's a few steps you have to do to  
> set it up).
> 
> >> Hence, my SVN checkouts are no longer  serialized -- which
> >> had been causing significant delays when the "test  get"
> >> phases of many MTT jobs were running at more-or-less the
> >> same time.
> >
> > If I see this:
> >
> > $ ls /foo
> > README.txt conf dav db format hooks locks
> >
> > How do I "svn checkout" from /foo using a "file:" URL?
> 
> svn co file:///foo
> 
> A file:// URL is just like an http[s]:// URL -- just put the absolute  
> directory name after it (to include the leading "/" -- so a file URL  
> will have 3 /'s).
> 

Ah. The *3* /'s are important. 2 slashes didn't do it :-)

-Ethan


> -- 
> Jeff Squyres
> Cisco Systems
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] MTT database access

[Ethan Mallove]

I thought maybe there was a reason Karol wanted separate
accounts that I hadn't thought of. Karol, which accounts do
you plan on using?

-Ethan


On Fri, Dec/14/2007 08:04:37PM, Jeff Squyres wrote:
> Do we really need all 3 database accounts for ubc?
> 
> 
> On Dec 14, 2007, at 5:52 PM, Ethan Mallove wrote:
> 
> > Hi Karol,
> >
> > I added three accounts:
> >
> > * ubc
> > * kmroz
> > * penoff
> >
> > OMPI MTT users use their organization name (e.g., "ubc" for
> > you folks), though your local UNIX username is recorded in
> > the results as well to help you sort through whose results
> > belong to who.
> >
> > Cheers,
> > Ethan
> >
> >
> > On Fri, Dec/14/2007 02:15:05PM, Karol Mroz wrote:
> >> Hi...
> >>
> >> I was wondering if it was possible to have 2 accounts (kmroz, penoff)
> >> added to the MTT database access list?
> >>
> >> Thanks.
> >>
> >> -- 
> >> Karol Mroz
> >> km...@cs.ubc.ca
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] MTT database access

[Ethan Mallove]

Hi Karol,

I added three accounts:

 * ubc
 * kmroz
 * penoff

OMPI MTT users use their organization name (e.g., "ubc" for
you folks), though your local UNIX username is recorded in
the results as well to help you sort through whose results
belong to who.

Cheers,
Ethan


On Fri, Dec/14/2007 02:15:05PM, Karol Mroz wrote:
> Hi...
> 
> I was wondering if it was possible to have 2 accounts (kmroz, penoff)
> added to the MTT database access list?
> 
> Thanks.
> 
> -- 
> Karol Mroz
> km...@cs.ubc.ca
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] unable to pull tests

[Ethan Mallove]

Do you have access to the ompi-tests repository?
What happens if you do this command outside of MTT?

  $ svn export https://svn.open-mpi.org/svn/ompi-tests/trunk/onesided

You could also try using "http", instead of "https" in
your svn_url.

-Ethan


On Wed, Nov/14/2007 02:04:27PM, Karol Mroz wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hello everyone...
> 
> I've been trying to get MTT set up to run tests on the SCTP BTL Brad
> Penoff and I (over at UBC) have developed. While I'm able to obtain a
> nightly  tarball, build and install the Open MPI middleware, I'm unable
> to pull any tests for the various svn repositories. Currently I've tried
> pulling IBM and Onesided tests as shown in the sample
> ompi-core-template.ini file.
> 
> Here is the output I see from the console when running with --verbose:
> - --
> *** MTT: ./mtt -f ../samples/ompi-core-template-kmroz.ini --verbose
> *** Reporter initializing
> *** Reporter initialized
> *** MPI get phase starting
> >> MPI get: [mpi get: ompi-nightly-trunk]
>Checking for new MPI sources...
>No new MPI sources
> *** MPI get phase complete
> *** MPI install phase starting
> >> MPI install [mpi install: gcc warnings]
>Installing MPI: [ompi-nightly-trunk] / [1.3a1r16706] / [gcc warnings]...
>Completed MPI install successfully
>Installing MPI: [ompi-nightly-trunk] / [1.3a1r16682] / [gcc warnings]...
>Completed MPI install successfully
>Installing MPI: [ompi-nightly-trunk] / [1.3a1r16723] / [gcc warnings]...
>Completed MPI install successfully
> *** MPI install phase complete
> *** Test get phase starting
> >> Test get: [test get: onesided]
>Checking for new test sources...
> - --
> 
> As you can see, MTT seems to hang on 'Checking for new test sources.'
> 
> I will attach a copy of the .ini file in hopes that someone may be able
> to point me in the right direction.
> 
> Thanks in advance.
> 
> 
> ompi-core-template-kmroz.ini:
> - ---
> # Copyright (c) 2006-2007 Cisco Systems, Inc.  All rights reserved.
> # Copyright (c) 2006-2007 Sun Microystems, Inc.  All rights reserved.
> #
> 
> # Template MTT configuration file for Open MPI core testers.  The
> # intent for this template file is to establish at least some loose
> # guidelines for what Open MPI core testers should be running on a
> # nightly basis.  This file is not intended to be an exhaustive sample
> # of all possible fields and values that MTT offers.  Each site will
> # undoubtedly have to edit this template for their local needs (e.g.,
> # pick compilers to use, etc.), but this file provides a baseline set
> # of configurations that we intend you to run.
> 
> # OMPI core members will need to edit some values in this file based
> # on your local testing environment.  Look for comments with "OMPI
> # Core:" for instructions on what to change.
> 
> # Note that this file is artificially longer than it really needs to
> # be -- a bunch of values are explicitly set here that are exactly
> # equivalent to their defaults.  This is mainly because there is no
> # reliable form of documentation for this ini file yet, so the values
> # here comprise a good set of what options are settable (although it
> # is not a comprehensive set).
> 
> # Also keep in mind that at the time of this writing, MTT is still
> # under active development and therefore the baselines established in
> # this file may change on a relatively frequent basis.
> 
> # The guidelines are as follows:
> #
> # 1. Download and test nightly snapshot tarballs of at least one of
> #the following:
> #- the trunk (highest preference)
> #- release branches (highest preference is the most recent release
> #  branch; lowest preference is the oldest release branch)
> # 2. Run all 4 correctness test suites from the ompi-tests SVN
> #- trivial, as many processes as possible
> #- intel tests with all_tests_no_perf, up to 64 processes
> #- IBM, as many processes as possible
> #- IMB, as many processes as possible
> # 3. Run with as many different components as possible
> #- PMLs (ob1, dr)
> #- BTLs (iterate through sm, tcp, whatever high speed network(s) you
> #  have, etc. -- as relevant)
> 
> #==
> # Overall configuration
> #==
> 
> [MTT]
> 
> # OMPI Core: if you are not running in a scheduled environment and you
> # have a fixed hostfile for what nodes you'll be running on, fill in
> # the absolute pathname to it here.  If you do not have a hostfile,
> # leave it empty.  Example:
> # hostfile = /home/me/mtt-runs/mtt-hostfile
> # This file will be parsed and will automatically set a valid value
> # for _max_np() (it'll count 

Re: [MTT users] "Passed" result on a run-time linker error

[Ethan Mallove]

On Sat, Oct/06/2007 07:59:43PM, Jeff Squyres wrote:
> A few questions:
> 
> - what's your "pass" criteria?  Is it looking at wifexited and  
> wexitstatus?

I have this: 

  pass = (_wexitstatus(), 0)

I'm guessing it should it be changed to this?

  pass = (_wifexited(), (_wexitstatus(), 0))

> - what does solaris return as an exit code in this case?
> 
>

Looks like 137.

  $ cc -G -o libbar.so bar.c
  $ file libbar.so
  libbar.so:  ELF 32-bit MSB dynamic lib SPARC32PLUS Version 1, V8+ 
Required, dynamically linked, not stripped
  $ cc foo.c -L`pwd` -R`pwd` -lbar -o foo   
 
  $ ldd foo
  libbar.so => /home/emallove/tmp/libbar.so
  ...
  $ ./foo
  $ echo $?
  0
  $ rm libbar.so
  $ ldd foo
  libbar.so => (file not found)
  ...
  $ ./foo
  ld.so.1: foo: fatal: libbar.so: open failed: No such file or directory
  Killed
  $ echo $?
  137

-Ethan


> On Sep 27, 2007, at 4:22 PM, Ethan Mallove wrote:
> 
> > Some trivial tests outputted the following:
> >
> >   ld.so.1: orterun: fatal: libopen-pal.so.0: open failed: No such  
> > file or directory
> >
> > See here:
> >
> >   http://www.open-mpi.org/mtt/index.php?do_redir=394
> >
> > How could these tests result in "pass"?
> >
> > I attached an mttdatabase debug file.
> >
> > -Ethan
> > 
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] Private-MTT setup HOWTOs

[Ethan Mallove]

Gil,

Josh created a few HOWTO pages that should help you get
your internal Mellanox MTT server setup:

  https://svn.open-mpi.org/trac/mtt/wiki/HTTPServer
  https://svn.open-mpi.org/trac/mtt/wiki/ServerMaintenance
  https://svn.open-mpi.org/trac/mtt/wiki/Database

Josh, another item Gil and I talked about was moving a
subset of test results from the internal Mellanox database
into the general Open MPI database. Could the scripts in
server/sql/support accomplish this?

-Ethan

Re: [MTT users] Problems running MTT with already installed MPICH-MX

[Ethan Mallove]

On Fri, Sep/28/2007 01:39:22PM, Tim Mattox wrote:
> This might also be a problem:
> > [MPI install: MPICH-MX]
> > mpi_get = mpich-mx
> > module = MPICH2
> 
> As far as I can tell, the install module should be something like this:
> module = Analyze::MPICH
> 
> But there isn't an MPICH module for that yet...  there are ones for OMPI,
> CrayMPI, HPMPI and IntelMPI.
> 
> The Analyze::OMPI module won't work, since it will try to use ompi_info...
> so you can lie and say it's one of the others, or...  I'm trying my own hacked
> Analyze::MPICH module to see if I understand how this works.
> 

It's okay to lie in this case :-) E.g., using Analyze::HPMPI
will just tell MTT that your MPI Install has C, C++, F77,
and F90 bindings so that it will proceed to running tests.
We should probably come up with a cleaner solution someday
though. Maybe an Install/Analyze/Generic.pm or something ...?

-Ethan


> On 9/28/07, Ethan Mallove <ethan.mall...@sun.com> wrote:
> > Hi Jelena,
> >
> > Change this line:
> >
> >   alreadyinstalled_dir = /usr/local/mpich-mx/bin
> >
> > To this:
> >
> >   alreadyinstalled_dir = /usr/local/mpich-mx
> >
> > Does that work?
> >
> > -Ethan
> >
> >
> > On Thu, Sep/27/2007 10:46:55PM, Jelena Pjesivac-Grbovic wrote:
> > > Hello,
> > >
> > > I am trying to test the MPICH-MX using MTT on our clusters and I am 
> > > hitting
> > > the wall.
> > > I was able to get Open MPI to run (did not try trunk yet - but nightly
> > > builds worked).
> > >
> > > The problem is that all phases seem to go through (including Test RUN and
> > > Reported) but nothing happens.
> > > I have attached the stripped down ini file (only with mpich-mx stuff)
> > > and output of the command
> > >
> > >./client/mtt --scratch /home/pjesa/mtt/scratch2 \
> > >--file
> > > /home/pjesa/mtt/collective-bakeoff/samples/ompi-core-perf-testing_mpich-mx.ini
> > > \
> > >--debug --verbose --print-time --no-section 'skampi imb osu'
> > >
> > > I think that it must be something stupid because the almost same script
> > > which downloads nightly open mpi build works.
> > >
> > > Thanks!
> > > Jelena
> > >  --
> > > Jelena Pjesivac-Grbovic -- Pjesa, Ph.D.
> > > Graduate Research Assistant
> > > Innovative Computing Laboratory
> > > Computer Science Department, UTK
> > > Claxton Complex 350
> > > (865) 974 - 6722 (865) 974 - 6321
> > > jpjes...@utk.edu
> > >
> > > "The only difference between a problem and a solution is that
> > >  people understand the solution."
> > >   -- Charles Kettering
> >
> > Content-Description: simplified ini file
> > > #==
> > > # Generic OMPI core performance testing template configuration
> > > #==
> > >
> > > [MTT]
> > > # Leave this string so that we can identify this data subset in the
> > > # database
> > > # OMPI Core: Use a "test" label until we're ready to run real results
> > > description = [testbake]
> > > #description = [2007 collective performance bakeoff]
> > > # OMPI Core: Use the "trial" flag until we're ready to run real results
> > > trial = 1
> > >
> > > # Put other values here as relevant to your environment.
> > > #hostfile = PBS_NODEFILE
> > > hostfile = /home/pjesa/mtt/runs/machinefile
> > >
> > > #--
> > >
> > > [Lock]
> > > # Put values here as relevant to your environment.
> > >
> > > #==
> > > # MPI get phase
> > > #==
> > >
> > > [MPI get: MPICH-MX]
> > > mpi_details = MPICH-MX
> > >
> > > module = AlreadyInstalled
> > > alreadyinstalled_dir = /usr/local/mpich-mx/bin
> > > alreadyinstalled_version = 1.2.7
> > >
> > > #module = Download
> > > #download_url = 
> > > http://www.myri.com/ftp/pub/MPICH-MX/mpich-mx_1.2.7..5.tar.gz
> > > ## You need to obtain the username and password from Myricom
&g

Re: [MTT users] Problem with reporter: selecting bitness

[Ethan Mallove]

This is a bug. Ticket to come. A workaround is to select
"Show" from the Show/Hide menu next to the Bitness
textfield when searching on Bitness.

Thanks,
Ethan


On Thu, Sep/20/2007 08:52:59AM, Tim Prins wrote:
> Hi,
> 
> I was doing a search and hit advanced, entered '32' into the bitness 
> field, and pressed submit. I got back the following error:
> 
> postgres: ERROR: operator does not exist: bit ~* "unknown" LINE 70: 
> (bitness ~* '32') ^ HINT: No operator matches the given name and 
> argument type(s). You may need to add explicit type casts. postgres: 
> ERROR: operator does not exist: bit ~* "unknown" LINE 70: (bitness ~* 
> '32') ^ HINT: No operator matches the given name and argument type(s). 
> You may need to add explicit type casts.
> 
> 
> Thanks,
> 
> Tim
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] OSU parsing module issue?

[Ethan Mallove]

The warnings emitted before the performance data *were* the
problem. Fixed in r1027.

-Ethan


On Mon, Sep/17/2007 04:06:26PM, Ethan Mallove wrote:
> Jeff,
> 
> Here is a run of "osu" test results that did not seem
> to register any latency_bandwidth data: 
> 
>   http://www.open-mpi.org/mtt/index.php?do_redir=340
> 
> Below is the mttdatabase debug file. Could the
> Analyze/OSU.pm module is getting flummoxed by the warnings
> that precede the actual performance data tables?
> 
>   $VAR1 = {
> 'duration_1' => '71 seconds',
> 'mpi_install_section_name_1' => 'developer',
> 'environment_2' => '',
> 'hostname' => 'fog01',
> 'result_stdout_1' => 'warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   warning:regcache incompatible with malloc
>   # OSU MPI Bandwidth Test (Version 2.1)
>   # Size  Bandwidth (MB/s) 
>   1   0.395771
>   2   0.782770
>   4   1.554345
>   8   2.796567
>   16  6.600534
>   32  13.161827
>   64  23.404541
>   128 43.974274
>   256 87.473876
>   512 161.514271
>   1024218.590781
>   2048224.491837
>   4096229.415259
>   8192238.724045
>   16384   243.492486
>   32768   244.995704
>   65536   245.507191
>   131072  232.206820
>   262144  201.486492
>   524288  182.285115
>   1048576 163.190881
>   2097152 170.839062
>   4194304 166.220132
>   ',
> 'launcher_1' => 'mpirun',
> 'launcher_3' => 'mpirun',
> 'resource_manager_2' => 'unknown',
> 'np_1' => '16',
> 'description_2' => undef,
> 'command_1' => 'mpirun -mca pml cm -mca mtl mx -x 
> LD_LIBRARY_PATH=/opt/mx/lib --host 
> fog01,fog02,fog03,fog04,fog05,fog06,fog07,fog08 -np 16 --prefix 
> /home/paklui/paklui-1.2-pml/shared-install osu_bw ',
> 'test_result_1' => 1,
> 'exit_value_3' => 0,
> 'result_stderr_2' => undef,
> 'network_1' => '',
> 'mpi_version' => '1.2.4b1r16122M',
> 'test_build_section_name_2' => 'osu',
> 'parameters_3' => '',
> 'result_stderr_3' => undef,
> 'duration_3' => '108 seconds',
> 'number_of_results' => '3',
> 'phase' => 'Test Run',
> 'exit_signal_3' => -1,
> 'parameters_2' => '',
> 'parameters_1' => '',
> 'start_timestamp_3' => 'Mon Sep 17 18:54:35 2007',
> 'variant_3' => 1,
> 'result_message_1' => 'Passed',
> 'test_name_1' => 'osu_bw',
> 'result_message_2' => 'Passed',
> 'result_message_3' => 'Passed',
> 'variant_2' => 1,
> 'fields' => 
> 'command,description,duration,environment,exit_signal,exit_value,launcher,mpi_install_section_name,mpi_name,mpi_version,network,np,parameters,phase,resource_manager,result_message,result_stderr,result_stdout,start_timestamp,test_build_id,test_build_section_name,test_name,test_result,variant',
> 'exit_value_2' => 0,
> 'exit_signal_2' => -1,
> 'test_name_2' => 'osu_latency',
> 'resource_manager_1' => 'unknown',
> 'command_3' => 'mpirun -mca pml cm -mca mtl mx -x 
> LD_LIBRARY_PATH=/opt/mx/lib --host 
> fog01,fog02,fog03,fog04,fog05,fog06,fog07,fog08 -np 16 --prefix 
> /home/paklui/paklui-1.2-pml/shared-install osu_bibw ',
> 'test_build_section_name_3' => 'osu',
> 'trial' => 0,
> 'mpi_name' => 'developer',
> 'duration_2' => '13 seconds',
> 'description_1' => undef,
> 'test_name_3' => 'osu_bibw',
> 'exit_value_1' =>

[MTT users] OSU parsing module issue?

[Ethan Mallove]

Jeff,

Here is a run of "osu" test results that did not seem
to register any latency_bandwidth data: 

  http://www.open-mpi.org/mtt/index.php?do_redir=340

Below is the mttdatabase debug file. Could the
Analyze/OSU.pm module is getting flummoxed by the warnings
that precede the actual performance data tables?

  $VAR1 = {
'duration_1' => '71 seconds',
'mpi_install_section_name_1' => 'developer',
'environment_2' => '',
'hostname' => 'fog01',
'result_stdout_1' => 'warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  # OSU MPI Bandwidth Test (Version 2.1)
  # SizeBandwidth (MB/s) 
  1 0.395771
  2 0.782770
  4 1.554345
  8 2.796567
  166.600534
  3213.161827
  6423.404541
  128   43.974274
  256   87.473876
  512   161.514271
  1024  218.590781
  2048  224.491837
  4096  229.415259
  8192  238.724045
  16384 243.492486
  32768 244.995704
  65536 245.507191
  131072232.206820
  262144201.486492
  524288182.285115
  1048576   163.190881
  2097152   170.839062
  4194304   166.220132
  ',
'launcher_1' => 'mpirun',
'launcher_3' => 'mpirun',
'resource_manager_2' => 'unknown',
'np_1' => '16',
'description_2' => undef,
'command_1' => 'mpirun -mca pml cm -mca mtl mx -x 
LD_LIBRARY_PATH=/opt/mx/lib --host 
fog01,fog02,fog03,fog04,fog05,fog06,fog07,fog08 -np 16 --prefix 
/home/paklui/paklui-1.2-pml/shared-install osu_bw ',
'test_result_1' => 1,
'exit_value_3' => 0,
'result_stderr_2' => undef,
'network_1' => '',
'mpi_version' => '1.2.4b1r16122M',
'test_build_section_name_2' => 'osu',
'parameters_3' => '',
'result_stderr_3' => undef,
'duration_3' => '108 seconds',
'number_of_results' => '3',
'phase' => 'Test Run',
'exit_signal_3' => -1,
'parameters_2' => '',
'parameters_1' => '',
'start_timestamp_3' => 'Mon Sep 17 18:54:35 2007',
'variant_3' => 1,
'result_message_1' => 'Passed',
'test_name_1' => 'osu_bw',
'result_message_2' => 'Passed',
'result_message_3' => 'Passed',
'variant_2' => 1,
'fields' => 
'command,description,duration,environment,exit_signal,exit_value,launcher,mpi_install_section_name,mpi_name,mpi_version,network,np,parameters,phase,resource_manager,result_message,result_stderr,result_stdout,start_timestamp,test_build_id,test_build_section_name,test_name,test_result,variant',
'exit_value_2' => 0,
'exit_signal_2' => -1,
'test_name_2' => 'osu_latency',
'resource_manager_1' => 'unknown',
'command_3' => 'mpirun -mca pml cm -mca mtl mx -x 
LD_LIBRARY_PATH=/opt/mx/lib --host 
fog01,fog02,fog03,fog04,fog05,fog06,fog07,fog08 -np 16 --prefix 
/home/paklui/paklui-1.2-pml/shared-install osu_bibw ',
'test_build_section_name_3' => 'osu',
'trial' => 0,
'mpi_name' => 'developer',
'duration_2' => '13 seconds',
'description_1' => undef,
'test_name_3' => 'osu_bibw',
'exit_value_1' => 0,
'start_timestamp_1' => 'Mon Sep 17 18:53:11 2007',
'environment_1' => '',
'result_stdout_3' => 'warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with malloc
  warning:regcache incompatible with 

Re: [MTT users] [MTT] #245: Support multiplicative effect for other INI params

[Ethan Mallove]

On Tue, Sep/04/2007 12:47:29PM, Jeff Squyres wrote:
> On Sep 4, 2007, at 10:57 AM, Ethan Mallove wrote:
> 
> > So if I have the following line in my [Test build: intel]
> > section, MTT will now build and run the Intel tests *twice*?
> > Once with compiler-A and once with compiler-B?
> >
> >   setenv = ("/path/to/compiler-A", "/path/to/compiler-B")
> 
> setenv is a bad example -- we can already take multiple values for  
> that and we do *not* create multiple runs for it.  For example:
> 
> setenv = FOO value1
> setenv = BAR value2
> 
> The mtt client is smart enough just to do 2 setenv's in a single  
> instance.

This ticket should be re-opened because the above case is
not covered. I *want* to get two separate test builds if I
give "prepend_path" a two-item list. Should we support two
kinds of environment INI parameter assignment? One for when
the user wants a multiplicative effect, and one for when
they do not? E.g.,

Do one test build. Prepend PATH with value1 and value2.

  prepend_path = < 
> I think that you can have some places that don't make sense to have  
> multiple values, and perhaps we should put in safeguards to ensure  
> that the INI file doesn't mistakenly do that.  For example:
> 
> module = (foo, bar)
> 
> That would be bad.  :-)
> 
> But this issue has always been around -- it's just that you can now do:
> 
> module = @foo@
> foo = (foo, bar)
> 
> 
> > -Ethan
> >
> >
> > On Tue, Sep/04/2007 10:03:09AM, Jeff Squyres wrote:
> >> I think we'll get it for everything because of the new @foo@ syntax.
> >>
> >> I don't think you'll want to use that effect everywhere, but it's now
> >> *possible* to do so.
> >>
> >>
> >> On Sep 4, 2007, at 9:57 AM, Ethan Mallove wrote:
> >>
> >>> So we will get a multiplicative effect on the following INI
> >>> parameters?
> >>>
> >>>   * setenv
> >>>   * unsetenv
> >>>   * env_module
> >>>
> >>> Any other parameters?
> >>>
> >>> -Ethan
> >>>
> >>>
> >>> On Fri, Aug/31/2007 10:18:30PM, MTT wrote:
> >>>> #245: Support multiplicative effect for other INI params
> >>>> --
> >>>> +-
> >>>>   Reporter:  emallove |   Owner:
> >>>>   Type:  enhancement  |  Status:  closed
> >>>>   Priority:  major|   Milestone:  v3.0
> >>>>  Component:  Client side  | Version:  trunk
> >>>> Resolution:  fixed|Keywords:  multiplicative exec INI
> >>>> --
> >>>> +-
> >>>> Changes (by jsquyres):
> >>>>
> >>>>   * status:  new => closed
> >>>>   * resolution:  => fixed
> >>>>
> >>>> Comment:
> >>>>
> >>>>  This is now done on the jms-new-parser branch.
> >>>>
> >>>> -- 
> >>>> Ticket URL: <https://svn.open-mpi.org/trac/mtt/ticket/ 
> >>>> 245#comment:2>
> >>>> MTT <http://www.open-mpi.org/>
> >>>> Issue tracking for the MPI Testing Tool.
> >>> ___
> >>> mtt-users mailing list
> >>> mtt-us...@open-mpi.org
> >>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>
> >>
> >> -- 
> >> Jeff Squyres
> >> Cisco Systems
> >>
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] Associating "description" with a single phase

[Ethan Mallove]

reporter.php has a search field for "description", but which
phase is that for? MPI Install, Test build, or Test run? A
scenario is that Sun will have a home-grown "funclet_files"
funclet set the "description" in the Install phase (e.g., a
label identifying which ISO is being tested). It would be
useful to have one of two things:

  * [easier] Have the "description" fields broken out in
reporter by phase. E.g.,
* MPI Install description
* Test build description
* Test run description
  * [harder] Be able to have a phase description carry over
to subsequent phases. I think the logic should work like
this: if an MPI Install has a description, then this
description will carry-over to its Test build and Test
runs if another description is not specified in those
post-MPI-Install phases. (Same for Test build.)

-Ethan

Re: [MTT users] [MTT] #245: Support multiplicative effect for other INI params

[Ethan Mallove]

So if I have the following line in my [Test build: intel]
section, MTT will now build and run the Intel tests *twice*?
Once with compiler-A and once with compiler-B?

  setenv = ("/path/to/compiler-A", "/path/to/compiler-B")

-Ethan


On Tue, Sep/04/2007 10:03:09AM, Jeff Squyres wrote:
> I think we'll get it for everything because of the new @foo@ syntax.
> 
> I don't think you'll want to use that effect everywhere, but it's now  
> *possible* to do so.
> 
> 
> On Sep 4, 2007, at 9:57 AM, Ethan Mallove wrote:
> 
> > So we will get a multiplicative effect on the following INI
> > parameters?
> >
> >   * setenv
> >   * unsetenv
> >   * env_module
> >
> > Any other parameters?
> >
> > -Ethan
> >
> >
> > On Fri, Aug/31/2007 10:18:30PM, MTT wrote:
> >> #245: Support multiplicative effect for other INI params
> >> -- 
> >> +-
> >>   Reporter:  emallove |   Owner:
> >>   Type:  enhancement  |  Status:  closed
> >>   Priority:  major|   Milestone:  v3.0
> >>  Component:  Client side  | Version:  trunk
> >> Resolution:  fixed|Keywords:  multiplicative exec INI
> >> -- 
> >> +-
> >> Changes (by jsquyres):
> >>
> >>   * status:  new => closed
> >>   * resolution:  => fixed
> >>
> >> Comment:
> >>
> >>  This is now done on the jms-new-parser branch.
> >>
> >> -- 
> >> Ticket URL: <https://svn.open-mpi.org/trac/mtt/ticket/245#comment:2>
> >> MTT <http://www.open-mpi.org/>
> >> Issue tracking for the MPI Testing Tool.
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] [MTT] #245: Support multiplicative effect for other INI params

[Ethan Mallove]

So we will get a multiplicative effect on the following INI
parameters? 

  * setenv
  * unsetenv
  * env_module

Any other parameters?

-Ethan


On Fri, Aug/31/2007 10:18:30PM, MTT wrote:
> #245: Support multiplicative effect for other INI params
> --+-
>   Reporter:  emallove |   Owner: 
>   Type:  enhancement  |  Status:  closed 
>   Priority:  major|   Milestone:  v3.0   
>  Component:  Client side  | Version:  trunk  
> Resolution:  fixed|Keywords:  multiplicative exec INI
> --+-
> Changes (by jsquyres):
> 
>   * status:  new => closed
>   * resolution:  => fixed
> 
> Comment:
> 
>  This is now done on the jms-new-parser branch.
> 
> -- 
> Ticket URL: 
> MTT 
> Issue tracking for the MPI Testing Tool.

[MTT users] WriteINI() calls in MPI/Install and Test/Build

[Ethan Mallove]

Do we still need to write out an INI file after each MPI
Install and Test Build? I thought we're using .dump files
instead? (I noticed this because it seemed to be taking a few
minutes to write the INI file after building the intel
tests.)

-Ethan

Re: [MTT users] Textfile Reporter

[Ethan Mallove]

Done. 

I brought it back as "ParsableTextfile" in both the trunk
and ompi-core-testers. You'll just have to do two things in
your INI file:

 * Change "module = Textfile" to "module = ParsableTextfile"
 * Rename "textfile_" params to "parsabletextfile_" params

Let me know if you run into any other issues with this.

-Ethan


On Tue, Jul/10/2007 02:27:27PM, Tim Prins wrote:
> Hmm, the INI file reporter does not seem to work for me. For the test results 
> I only get the information about the last test run. 
> 
> Anyways, I like the idea of pulling the data directly in from perl output but 
> just don't have the time to mess with it right now. For me bringing back the 
> old reporter would be easiest for the time being. However, I also need the 
> following patch applied to resurect a couple output fields that were removed 
> which we need:
> 
> Index: lib/MTT/Test/Analyze/Correctness.pm
> ===
> --- lib/MTT/Test/Analyze/Correctness.pm (revision 737)
> +++ lib/MTT/Test/Analyze/Correctness.pm (working copy)
> @@ -53,6 +53,8 @@
> 
>  test_name => $run->{name},
>  command => $run->{cmd},
> +test_build_section_name => $run->{test_build_simple_section_name},
> +
>  np => $run->{np},
>  exit_value => MTT::DoCommand::exit_value($results->{exit_status}),
>  exit_signal => MTT::DoCommand::exit_signal($results->{exit_status}),
> Index: lib/MTT/MPI/Install.pm
> ===
> --- lib/MTT/MPI/Install.pm  (revision 737)
> +++ lib/MTT/MPI/Install.pm  (working copy)
> @@ -505,6 +505,8 @@
>  my $report = {
>  phase => "MPI Install",
> 
> +mpi_install_section_name => $config->{simple_section_name},
> +
>  bitness => $config->{bitness},
>  endian => $config->{endian},
>  compiler_name => $config->{compiler_name},
> 
> 
> Thanks,
> 
> Tim
> 
> On Tuesday 10 July 2007 11:46:34 am Ethan Mallove wrote:
> > Whoops! I didn't realize anyone was using that Textfile
> > module. We can resurrect that if you'd like (call it
> > ParseableTextfile).
> >
> > There's also the INIFile Reporter. That might be your best
> > bet, since there's a Config::INIFiles CPAN module. (Your
> > wrappers are in Perl, right?) Though wouldn't it be even
> > easier if there were a PerlDumper Reporter module so you
> > could read in the data *directly* to your Perl wrappers?
> > Your wrapper would do no parsing then. E.g.,
> >
> > open(FILE, "< $file");
> > undef $/;
> > $mtt_results = ;
> >
> > -Ethan
> >
> > On Mon, Jul/09/2007 06:07:51PM, Tim Prins wrote:
> > > Hi,
> > >
> > > With the new version of MTT, the textfile report file
> > > format changed to a more human readable format. Since we
> > > here at IU use a script to parse this, it presents a bit
> > > of a problem. I can update our script, but was wondering
> > > how stable this new output format is.
> > >
> > > If it will not be very stable, I was wondering if the
> > > developers would consider adding a parseable textfile
> > > output module. The easiest thing to do for this would be
> > > to just import the old textfile module as a new parseable
> > > module. I have tried this and it seems to work fine,
> > > however there may be problems with this that I am unaware
> > > of.
> > >
> > > I can deal with this either way, but just thought it might
> > > make things easier to have a parseable format that is
> > > relatively static, and a human readable format that can be
> > > tweaked for useability as time goes by.
> > >
> > > Thanks,
> > >
> > > Tim
> > > ___
> > > mtt-users mailing list
> > > mtt-us...@open-mpi.org
> > > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Perl Wrap Error

[Ethan Mallove]

On Fri, Jul/06/2007 01:22:06PM, Joshua Hursey wrote:
> Anyone seen the following error from MTT before? It looks like it is in the 
> reporter stage.
>
> <->
> shell$ /spin/home/jjhursey/testing/mtt//client/mtt --mpi-install  --scratch 
> /spin/home/jjhursey/testing/scratch/20070706 --file 
> /spin/home/jjhursey/testing/etc/jaguar/simple-svn.ini --print-time 
> --verbose --debug 2>&1 1>> 
> /spin/home/jjhursey/testing/scratch/20070706/output.txt
> This shouldn't happen at /usr/lib/perl5/5.8.3/Text/Wrap.pm line 64.
> shell$
> <->

"This shouldn't happen at ..." is the die message?

Try this INI [Reporter: TextFile] section:

{{{
  [Reporter: text file backup]
  module = TextFile

  textfile_filename = $phase-$section-$mpi_name-$mpi_version.txt

  # User-defined report headers/footers
  textfile_summary_header = <
> The return code is: 6400
>
> I attached the output log incase that helps, and the INI file.
>
> -- Josh

Re: [MTT users] Database insert errors

[Ethan Mallove]

Hi Tim,

I see some of these FOREIGN KEY constraint errors every
night. There's a system of speedy and archive tables to keep
short-term queries fast, but it has bugs. There are rows in
the archive tables that should be mirrored in the speedy
tables, but this is not always the case. We (well, mostly
Josh) are working on an improved system of "partitioning"
the huge Postgres tables to keep queries fast which will
hopefully also resolve these referential integrity problems.

-Ethan


On Sun, Jul/01/2007 09:37:17PM, Tim Prins wrote:
> Hi Folks,
> 
> For a while now we have been getting errors when MTT tries to submit its test 
> results to the database. The weird thing is that it only happens on our 1.2 
> runs, not our trunk runs. 
> 
> Here is the first few lines of the error output:
> *** WARNING: MTTDatabase server notice: fields is not in mtt3 database.
> MTTDatabase server notice: test_build_section_name is not in mtt3
> database.
> MTTDatabase server notice: mpi_install_section_name is not in mtt3
> database.
> MTTDatabase server notice: mtt_version_minor is not in mtt3 database.
> MTTDatabase server notice: stop_timestamp is not in mtt3 database.
> MTTDatabase server notice: mtt_version_major is not in mtt3 database.
> MTTDatabase server notice: number_of_results is not in mtt3 database.
> MTTDatabase server notice: test_run_section_name is not in mtt3
> database.
> 
> MTT submission for test run
> MTTDatabase server error:
> SQL QUERY:
>  INSERT INTO speedy_test_run
>  (np,
> variant,
> test_build_id,
> command,
> test_name,
> test_run_id)
>  VALUES
>  ('8',
> '1',
> '20809',
> 'mpirun  -mca pml ob1 -mca btl_tcp_if_include eth0 -mca btl
> tcp,sm,self -np 8 --prefix
> /san/homedirs/mpiteam/mtt-runs/thor/20070630-Nightly/pb_0/installs/k1mL
> /install collective/allgather ',
> 'allgather',
> '14517807')
> 
> SQL ERROR: ERROR:  insert or update on table "speedy_test_run" violates
> foreign key constraint "$1"
> DETAIL:  Key (test_build_id)=(20809) is not present in table
> "speedy_test_build".
> 
> Another strange thing is that the output says that the build information and 
> some test results have been submitted, but I do not see them in the reporter. 
> Any ideas?
> 
> Thanks,
> 
> Tim
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] max_np on multi-processor nodes

[Ethan Mallove]

env_max_procs uses this order of precedence to get a
"max_np".

  1. Various resource managers
  2. If a global hostfile was specified
  3. If a global hostlist was specified
  4. If a global max_np was specified

Shouldn't 4 be at the top of the list?

I'm doing single node testing, but the fact that I have only
one node listed in hostlist causes env_max_procs to return
"1". I'd like to override the 1 value with the number of
CPU's on the single node I'm running on.

-Ethan

Re: [MTT users] funclet and quoting

[Ethan Mallove]

On Tue, Jun/12/2007 04:53:21PM, Jeff Squyres wrote:
> On Jun 12, 2007, at 10:50 AM, Ethan Mallove wrote:
> 
> >>> eval $btls = MTT::Functions::if(MTT::Functions::regexp
> >>> (MTT::Functions::shell("hostname"), "v440-2|v20z-2"),
> >>>  MTT::Functions::enumerate("self,sm,tcp"),
> >>>  MTT::Functions::if("MTT::Functions::regexp
> >>> ("MTT::Functions::shell("hostname"), "v440-6"),
> >>>  MTT::Functions::enumerate("udapl"),
> >>>  MTT::Functions::enumerate("sm", "tcp",
> >>> "sm,tcp", "udapl", "sm,udapl")
> >>>  )
> >>>  )
> >>
> >> Ah -- I had to think about this a bit before I figured it out/
> >> remembered.  I think we won't get the expansion effect if you do it
> >> this way.
> >
> > No expansion effect? As long as the perl code evaluates to
> > an array, couldn't it to work? E.g.,
> 
> We'll need to revamp all the current funclets to accept
> arrays and  return [potentially] arrays of arrays.  What
> if, as a  counterproposal, we have a () funclet?
> Then you could do the  following:
> 
> 
> 
> btls = ("if ($hostname =~ /foo/) { \
> return ("self,sm,tcp");  \
> } elsif ($hostname =~ /bar/) {   \
> return ("udapl");\
> } else { \
> return ("sm", "tcp", "sm,tcp", "udapl", "sm,udapl"); \
> }")
>
> 
> 
> (or something like that)
>

I like that. Is that in anticipation of having (),
(), (), (), (), ... ?  To reduce some
clutter, maybe a param like this would be fun, e.g.,

  default_ini_evaluator = perl

So MTT will assume you mean () if there's
no leading '&'.

Other legal options being ...

  default_ini_evaluator = python
  default_ini_evaluator = sh
  default_ini_evaluator = tcl
  default_ini_evaluator = ...

-Ethan


> 
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] funclet and quoting

[Ethan Mallove]

On Tue, Jun/12/2007 08:07:20AM, Jeff Squyres wrote:
> On Jun 11, 2007, at 6:02 PM, Ethan Mallove wrote:
> 
> > btls = ("("("hostname")", "v440-2|v20z-2")", \
> >"("self,sm,tcp")", \
> >"("("("hostname")", "v440-6")", \
> >"("udapl")", \
> >"("sm", "tcp", "sm,tcp", "udapl",  
> > "sm,udapl")"" \
> >) \
> >)
> 
> Yowza!  We really need to make this quoting situation better  
> somehow.  :-(
> 
> Don't you have an extra ) in there at the end?
> 
> > How come EvaluateString() doesn't translate this:
> >
> > btls = ("("("hostname")", "v440-2|v20z-2")", \
> >"("self,sm,tcp")", \
> >"("("("hostname")", "v440-6")", \
> >"("udapl")", \
> >"("sm", "tcp", "sm,tcp", "udapl",  
> > "sm,udapl")"" \
> >) \
> >)
> >
> > Into one big eval block like this:
> >
> > eval $btls = MTT::Functions::if(MTT::Functions::regexp 
> > (MTT::Functions::shell("hostname"), "v440-2|v20z-2"),
> >  MTT::Functions::enumerate("self,sm,tcp"),
> >  MTT::Functions::if("MTT::Functions::regexp 
> > ("MTT::Functions::shell("hostname"), "v440-6"),
> >  MTT::Functions::enumerate("udapl"),
> >  MTT::Functions::enumerate("sm", "tcp",  
> > "sm,tcp", "udapl", "sm,udapl")
> >  )
> >  )
> >
> > So we could put the burden of the nasty recursive
> > parenthesis/quote parsing on perl, rather than MTT. That
> > would seem to naturally make the INI funclet syntax more
> > Perl-like.
> 
> Ah -- I had to think about this a bit before I figured it out/ 
> remembered.  I think we won't get the expansion effect if you do it  
> this way.  

No expansion effect? As long as the perl code evaluates to
an array, couldn't it to work? E.g.,

  exec = mpirun --mca btl @btls@ ...

  # Returns an array
  btls = ("sm", "tcp")

So EvaluateString() would do only this:

  * Substitute all () with MTT::Functions::funcs()
  * Substitute all $vars with same-INI-section params
  * Backslash escape @ symbols so they show up as strings in
the eval'd result
  * Perform an "eval" on the final string
  * Iterate over the result of the above step's string,
replacing "@multiplier" strings with the corresponding
element of the @multiplier array 

> Although it might solve some other issues (e.g., https:// 
> svn.open-mpi.org/trac/mtt/ticket/184)...?
>

The mess atop this email would simplify to:

btls =  if ($hostname =~ /foo/) {\
return ("self,sm,tcp");  \
} elsif ($hostname =~ /bar/) {   \
return ("udapl");\
} else { \
return ("sm", "tcp", "sm,tcp", "udapl", "sm,udapl"); \
}

-Ethan


> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] New version of MTT

[Ethan Mallove]

Folks,

MTT version 2.1 is live! Please do an "svn up" at your
earliest convenience. As always, report any issues you
encounter to this mailing list.

Thanks,
Ethan


On Wed, May/09/2007 09:09:46PM, Jeff Squyres wrote:
> All --
> 
> We are just about to release MTT version 2.1.  Woo hoo!
> 
> There are a bunch of new features available -- see this
> wiki page for  details:
> 
>  https://svn.open-mpi.org/trac/mtt/wiki/News-09-May-2007
> 
> We strongly encourage everyone to update their INI files
> for new  "pass" and "skip" criteria with the new _*()
> funclets that are  available (examples are on the wiki
> page).
> 
> Ethan will reply shortly with the timeframe for when we
> plan to cut  the new version over on the
> "ompi-core-testers" branch.
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] MTT server error (user: ibm)

[Ethan Mallove]

Camille,

Your results today were borked somehow.  What options did
you give to client/mtt?  *Something* happened with the
--trial option.  I patched the MTT client, though I've never
seen this error before.  Unfortunately, you would need to
re-run your tests (after doing "svn up") to submit them to
the DB :(

-Ethan


- Forwarded message from ethan.mall...@sun.com -

From: ethan.mall...@sun.com
List-Post: mtt-users@lists.open-mpi.org
Date: Fri, 09 Mar 2007 12:31:25 -0500
To: ethan.mall...@sun.com
Subject: MTT server error (user: ibm)
Sender: apa...@osl.iu.edu
Reply-to: ethan.mall...@sun.com
Original-recipient: rfc822;ethan.mall...@sun.com



SQL QUERY: 
 INSERT INTO speedy_results 
 (trial,
merge_stdout_stderr,
test_result,
phase,
exit_signal,
exit_value,
client_serial,
result_message,
result_stderr,
result_stdout,
environment,
start_timestamp,
submit_timestamp,
duration,
submit_id,
phase_id,
results_id) 
 VALUES 
 ('1 ',
DEFAULT,
'1',
'3',
'-1',
'0',
'7428',
'Passed',
'',
'Hello, C++ world!  I am 0 of 2
Hello, C++ world!  I am 1 of 2
',
DEFAULT,
'Fri Mar  9 16:50:05 2007',
'Fri Mar  9 17:31:37 2007',
'2 seconds',
'564',
'3360557',
'3469545') 

SQL ERROR: ERROR:  invalid input syntax for type boolean: "1 "
SQL ERROR: 

- End forwarded message -

Re: [MTT users] script to resubmit results

[Ethan Mallove]

I just added a client/mtt-submit script to the mtt/trunk SVN
repository which will help you.  You'll need to use it like
this:

$ client/mtt-submit -f '/directory/where/results/are/*.txt' -g -u cisco -p 


/directory/where/results/are/ is specified in your INI file
as mttdatabase_debug_filename.

CAUTION: Use this script with care!  For example, you could
end up with duplicate test results in the database if you
used mtt-submit twice on the same file.

-Ethan


On Thu, Mar/01/2007 08:40:17AM, Bill D'amico wrote:
> 
>We had a proxy server fail during the night and may have had some
>results get lost. Jeff asked me to check for existence of a
>resubmission script - is there one, and if so where is it?
> 
>--
>Bill D'Amico
>[1]bdam...@cisco.com
>Server virtualization Business Unit
> 
> References
> 
>1. mailto:bdam...@cisco.com

> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] Deadline to go back to "testers" branch

[Ethan Mallove]

Folks,

The new branch has been cut. Please switch back to the
ompi-core-testers branch by Friday.  The SVN URL is
https://svn.open-mpi.org/svn/mtt/branches/ompi-core-testers.
The MTT 2.0 server-side scripts are located (again) at
http://www.open-mpi.org/mtt, which means you need to update
the mttdatabase_url parameter in your INI to be
http://www.open-mpi.org/mtt/submit/. (Careful, that trailing
'/' is needed!) Lastly, the Reporter is here
http://www.open-mpi.org/mtt/reporter.php.

Thanks!
Ethan

Re: [MTT users] [devel-core] first steps with MTT

[Ethan Mallove]

Hi Andreas,

mtt-us...@open-mpi.org is where you want to send MTT-related
stuff.

On Tue, Jan/23/2007 09:12:59AM, Andreas Kn?pfer wrote:
> Hello everyone,
> 
> After our first experiments with MTT I'd like to ask a
> couple of questions. I guess, if I put them here the
> answers will go to the list archive as well.  maybe I can
> write a wiki page for the first steps towards MTT, too.
> 
> This leads me to the first question? what's the
> login/passwd for the wiki?  And how about the central data
> base for results? probably that answers should not go to
> the list.
>

There's anonymous read-only access to the wiki.

https://svn.open-mpi.org/trac/mtt/wiki

You need an HTTP account for sending up results to the
database. username's are organization names (e.g., cisco,
sun, lanl, etc.). What do you want as your username?
"tu-dresden" or "zih"?

> So far I'm using text output only. How can I set up an own
> data base? There are some PHP files in the MTT tarball. Do
> they initialize data base and tables and so on? is there
> any script for this? Does is require mysql or postgres or
> any of the two?
>

Postgres, but the server-side is already taken care of.
There's a centralized database setup at Indiana Univ, and
the MTT test results can be viewed here:
http://www.open-mpi.org/mtt.


> When I did my own test scripts for a simple test
> application I followed the example INI files provided and
> some pieces of documentation and the Perl code. Is there a
> "real" documentation about all bells and whistles? If not,
> could we write one or at least provide a step-by-step
> guide for beginners?

The wiki is a good starting point. There's links to a bunch
of stuff on the MTT home wiki page here: 

https://svn.open-mpi.org/trac/mtt/wiki

The closest thing to a Beginner's Guide to using the MTT
client would be:

https://svn.open-mpi.org/trac/mtt/wiki/MTTOverview

> 
> When I want to do some tests whether a test run was
> successful of not, which way would you suggest. Either
> doing it in bash commands placed in the INI file or rather
> write new Perl modules that will be specific to my
> application? The trivial test case does part of both,
> doesn't it?
>

B. new Perl modules :)

But the test suites that are listed in the
samples/ompi-core-templates.ini file already have modules in
place to Get (e.g., svn export), Build (e.g., make), Run
(mpirun), and Analyze (prepare test results for the
database).

If you add a *new* test suite to ompi-tests, the existing
modules should be able to do all of the above (with the
exception of Analyze, if the test outputs performance data).
You just might need to add some special conditions in the [Test
run: foo] section of the INI file to indicate e.g., whether
some tests are *supposed* to fail, some tests take longer
time span to run, etc.


> Furthermore, there is a caching scheme for test results
> between successive runs. MTT avoids test that were run
> before. Where does it store this information? How to
> explicitly clear it?
> 

The cached data is stored as serialized Perl variables
down in the scratch directory.

The --force option "forces" MTT to run everything,
regardless of whether it's been run before.

> 
> As one of our current goals, I'm integrating VampirTrace
> into the MTT test cycle. The most straight forward way
> would be to have a test application tarball which includes
> VampirTrace sources, wouldn't it? Would it be feasible to
> have two download clauses in a single test-get phase and
> two successive build clauses in a single test-build
> phase? Could this work with the current macros?
>

I don't know of a user that's done two fetches/compiles in a
single section yet. Why do you need to download/build twice
for a single test suite?  (There's nothing preventing this,
just wondering.) I think what you want would look something
like this:

[Test get: VampirTrace]
url1 = http://www.foo.com
url2 = http://www.bar.com
module = Tarball

[Test build: VampirTrace]
module = Shell
shell_build_command = < 
> Thank you very much for your answers! 
> Andreas
> 
> -- 
> Dipl. Math. Andreas Knuepfer, 
> Center for Information Services and 
> High Performance Computing (ZIH), TU Dresden, 
> Willersbau A114, Zellescher Weg 12, 01062 Dresden
> phone +49-351-463-38323, fax +49-351-463-37773
> ___
> devel-core mailing list
> devel-c...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel-core

Re: [MTT users] Minor bug found in MTT 2 client side.

[Ethan Mallove]

Tim,

I wonder if reporter.php can help out some here. IIRC, a
major advantage of your parsing scripts was to see results
of individual MTT invocations, and characteristics of
invocations that reporter.php wasn't able to show.
reporter.php now keeps track of results ''by invocation''
much more systematically then before.  Could you give me a
use case of what you want you want in a report? (Or point me
to some examples in ompi-tests/trunk/iu/mtt - Josh had some
mock ups of your reports, but I lost track of them.)

-Ethan


On Fri, Jan/19/2007 11:50:54AM, Jeff Squyres wrote:
> On Jan 19, 2007, at 8:55 AM, Tim Mattox wrote:
> 
> > Sorry, I wasn't clear.  The current output for each test
> > in the  debug file usually includes a line "test_result:
> > X" with X replaced by a number.  However, for tests that
> > fail outright, this line is missing.  This  missing line
> > happened to correspond to the tests that had a newline
> > in the  result message that I discussed (snipped) above.
> >
> > Please don't put in the parentheses things.  That was just me  
> > commenting
> > on which number meant what.
> 
> I forgot to mention -- let me know where you land on the whole  
> "continue to parse debug output" vs. "write an MTT reporter" issue.   
> If you write a new reporter, this particular issue ^^ is moot.
> 
> -- 
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Minor bug found in MTT 2 client side.

[Ethan Mallove]

On Fri, Jan/19/2007 08:55:26AM, Tim Mattox wrote:
> Hi All,
> 
> On 1/18/07, Jeff Squyres  wrote:
> > On Jan 18, 2007, at 10:37 PM, Tim Mattox wrote:
> [snip description of a newline bug]
> 
> > Fixed -- none have newlines now, so they'll all be in the one-line
> > output format.
> 
> Thanks.
> 
> > > I don't know if it is related or not, but for tests that fail without
> > > timing out,
> > > the debug output from MTT for that test does NOT have a line like
> > > these:
> > > test_result: 1  (passed)
> > > test_result: 2  (skipped)
> > > test_result: 3  (timed out)
> >
> > Are you sure?  Lines 80-84 of Correctness.pm are:
> >
> >  if ($results->{timed_out}) {
> >  Warning("$str TIMED OUT (failed)\n");
> >  } else {
> >  Warning("$str FAILED\n");
> >  }
> >
> > Are you talking about some other output?  Or are you asking for
> > something in (parentheses)?
> 
> Sorry, I wasn't clear.  The current output for each test in the debug file
> usually includes a line "test_result: X" with X replaced by a number.
> However, for tests that fail outright, this line is missing.  This missing
> line happened to correspond to the tests that had a newline in the result
> message that I discussed (snipped) above.
> 
> Please don't put in the parentheses things.  That was just me commenting
> on which number meant what.
> 
> 
> > If you're in the middle of revamping your parser to match the MTT 2.0
> > output, I might suggest that it might be far easier to just
> > incorporate your desired output into MTT itself, for two reasons:
> >
> > 1. the debug output can change at any time; it was meant to be for
> > debugging, not necessarily for screen scraping.
> 
> Point taken.
> 
> > 2. there would be no need for screen scraping/parsing; you would have
> > the data immediately available and all you have to do is output it
> > into the format that you want.  We should be able to accommodate
> > whatever you need via an MTT Reporter plugin.  I'm guessing this
> > should be pretty straightforward...?
> 
> Where can I find some documentation for or examples of a plugin?

They're all in this directory:

https://svn.open-mpi.org/trac/mtt/browser/trunk/lib/MTT/Reporter/

You probably want to look at these:

https://svn.open-mpi.org/trac/mtt/browser/trunk/lib/MTT/Reporter/TextFile.pm
https://svn.open-mpi.org/trac/mtt/browser/trunk/lib/MTT/Reporter/Email.pm
https://svn.open-mpi.org/trac/mtt/browser/trunk/lib/MTT/Reporter/INIFile.pm

-Ethan

> 
> > --
> > Jeff Squyres
> > Server Virtualization Business Unit
> > Cisco Systems
> 
> 
> -- 
> Tim Mattox - http://homepage.mac.com/tmattox/
>  tmat...@gmail.com || timat...@open-mpi.org
> I'm a bright... http://www.the-brights.net/
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

[MTT users] "Cherry picking" feature

[Ethan Mallove]

Folks,

I forgot to mention on Tuesday's teleconference, a useful
feature in Performance mode. If you click Performance, above
the graphs is a summary with a far left column containing
checkboxes. Those checkboxes allow you to ''cherry pick''
which rows to plot in the below graphs. So you can
check/uncheck them and then click the ''cherry'' button at
the foot of the table to prune out line plots. Test it out
here:

http://www.open-mpi.org/~emallove/mtt/reporter.php?do_redir=62

-Ethan

Re: [MTT users] Corrupted MTT database or incorrucet query

[Ethan Mallove]

I can infer that you have an MPI Install section labeled
"odin 64 bit gcc". A few questions:

* What is the mpi_get for that section (or how does that
  parameter get filled in by your automated scripts)?  
* Do you start with a fresh scratch tree every run?
* Could you email me your scratch/installs/mpi_installs.xml
  files?

I checked on how widespread this issue is, and found that
18,700 out of 474,000 Test Run rows in the past month have a
mpi_version/command (v1.2-trunk) mismatch. Occuring in both
directions (version=1.2, command=trunk and vice versa).
They occur on these clusters:

 Cisco MPI development cluster
 IU Odin
 IU - Thor - TESTING

There *is* that race condition in which one mtt submitting
could overwrite another's index. Do you have "trunk" and
"1.2" runs submitting to the database at the same time?


On Sun, Nov/12/2006 06:04:17PM, Jeff Squyres (jsquyres) wrote:
> 
>I feel somewhat better now.  Ethan - can you fix?
> -Original Message-
>From:   Tim Mattox [[1]mailto:timat...@open-mpi.org]
>Sent:   Sunday, November 12, 2006 05:34 PM Eastern Standard Time
>To: General user list for the MPI Testing Tool
>Subject:[MTT users] Corrupted MTT database or incorrucet query
>Hello,
>I just noticed that the MTT summary page is presenting
>incorrect information for our recent runs at IU.  It is
>showing failures for the 1.2b1 that actaully came from
>the trunk!  See the first entry in this table:
>http://www.open-mpi.org/mtt/reporter.php?_start_test_timestamp=200
>6-11-12%2019:12:02%20through%202006-11-12%2022:12:02_platform_id=co
>ntains_platform_id=IU_phase=runs_success=fail_atom=*by_t
>est_case=Table_agg_timestamp=-_mpi_name=All_mpi_version
>=All_os_name=All_os_version=All_platform_hardware=All_
>platform_id=All_platform_id=off&1-page=off_bookmarks_bookmar
>ks
>Click on the [i] in the upper right (the first entry)
>to get the popup window which shows the MPIRrun cmd as:
>mpirun -mca btl tcp,sm,self -np 6 --prefix
>/san/homedirs/mpiteam/mtt-runs/odin/20061112-Testing-NOCLN/parallel-bl
>ock-3/installs/ompi-nightly-trunk/odin_64_bit_gcc/1.3a1r12559/install
>dynamic/spawn Note the path has "1.3a1r12559" in the
>name... it's a run from the trunk, yet the table showed
>this as a 1.2b1 run.  There are several of these
>missattributed errors.  This would explain why Jeff saw
>some ddt errors on the 1.2 brach yesterday, but was
>unable to reproduce them.  They were from the trunk!
>--
>Tim Mattox - [2]http://homepage.mac.com/tmattox/
> tmat...@gmail.com || timat...@open-mpi.org
>I'm a bright... [3]http://www.the-brights.net/
>___
>mtt-users mailing list
>mtt-us...@open-mpi.org
>[4]http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> References
> 
>1. mailto:timat...@open-mpi.org
>2. http://homepage.mac.com/tmattox/
>3. http://www.the-brights.net/
>4. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users


-- 
-Ethan

Re: [MTT users] [MTT bugs] [MTT] #132: Print BIG WARNING in client if MTTDatabase fails to submit properly

[Ethan Mallove]

On Fri, Nov/10/2006 11:11:39PM, MTT wrote:
> #132: Print BIG WARNING in client if MTTDatabase fails to submit properly
> --+-
>   Reporter:  jsquyres |   Owner:  emallove
>   Type:  defect   |  Status:  new 
>   Priority:  blocker  |   Milestone:  v1.1
>  Component:  Client side  | Version:  trunk   
> Resolution:   |Keywords:  
> --+-
> Old description:
> 
> > Add some intelligence in submit.php so that it knows when an insert into
> > the database has failed.
> >
> > Somehow convey back to the client that one or more inserts has failed so
> > that the client can print out a '''BIG HAIRY WARNING''' that the results
> > did not get successfully insertted.  When this happens, automatically
> > drop a database debug file containing all the data (so that it isn't
> > lost) and include its filename in the '''BIG HAIRY WARNING.'''  This file
> > can then be analyzed later for a post mortem, etc.  Be sure that this
> > file is ''always'' dropped when there's a database insert error,
> > regardless of what the INI file setting is for dropping database debug
> > files.
> >
> > Bonus points will be awarded if submit.php can communicate back to the
> > client ''which'' results failed to submit, but that's not a huge deal
> > (we'll be able to do this better in 2.0).
> >
> > The '''BIG HAIRY WARNING''' should probably be output by the client both
> > when it happens and at the very end of the run so that if all you see in
> > the morning is the end of the MTT run on the screen, the warning is there
> > and the user don't have to search through the output just to know if
> > their data submitted properly or not.
> 
> New description:
> 
>  Add some intelligence in submit.php so that it knows when an insert into
>  the database has failed.
> 
>  Somehow convey back to the client that one or more inserts has failed so
>  that the client can print out a '''BIG HAIRY WARNING''' that the results
>  did not get successfully insertted.  When this happens, automatically drop
>  a database debug file containing all the data (so that it isn't lost) and
>  include its filename in the '''BIG HAIRY WARNING.'''  This file can then
>  be analyzed later for a post mortem, etc.  Be sure that this file is
>  ''always'' dropped when there's a database insert error, regardless of
>  what the INI file setting is for dropping database debug files.
> 
>  Bonus points will be awarded if submit.php can communicate back to the
>  client ''which'' results failed to submit, but that's not a huge deal
>  (we'll be able to do this better in 2.0).
> 
>  The '''BIG HAIRY WARNING''' should probably be output by the client both
>  when it happens and at the very end of the run so that if all you see in
>  the morning is the end of the MTT run on the screen, the warning is there
>  and the user don't have to search through the output just to know if their
>  data submitted properly or not.

I'm used grepping on the --debug output from client/mtt, which means e.g., I
will now see double the number of POSTGRES ERRORs. Isn't that confusing? At the
*end* of the mtt run, I'd much prefer to just say, e.g., "34 SQL errors
occured". If they want to see the actual SQL error messages from submit.php,
users can do:

$ client/mtt --debug ... | tee debug && grep -i POSTGRES debug

If submit.php?debug is used, the exact submission entry that caused the error(s)
can be traced.

> 
>  The should be done on the trunk and the release branch; the code will
>  likely be at least slightly different between the two, but quite similar.
> 
> -- 
> Ticket URL: 
> MTT 
> Issue tracking for the MPI Testing Tool.


-- 
-Ethan

Re: [MTT users] New MTT home page

[Ethan Mallove]

How's it now?

[P] Passes
[F] Failures
[*] Passes and Failures


On Fri, Nov/10/2006 01:01:37PM, Tim Mattox wrote:
> Hi Ethan,
> These look great!  Can you add one more column of choices with the
> heading "Failed Test Runs" which would be the same as "Test Runs",
> but without the entries that had zero failures.
> 
> If it wouldn't be too much trouble, could you also add a "Past 48
> Hours" section,
> but this is lower priority than adding a "Failed Test Runs" column.
> 
> Thanks!
> 
> On 11/10/06, Ethan Mallove <ethan.mall...@sun.com> wrote:
> > Folks,
> >
> > The MTT home page has been updated to display a number of
> > quick links divided up by date range, organization, and
> > testing phase (MPI installation, test build, and test run).
> > The custom and summary reports are linked from there.
> >
> > http://www.open-mpi.org/mtt/
> >
> > --
> > -Ethan
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >
> 
> 
> -- 
> Tim Mattox - http://homepage.mac.com/tmattox/
>  tmat...@gmail.com || timat...@open-mpi.org
> I'm a bright... http://www.the-brights.net/
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

-- 
-Ethan

[MTT users] 1.2b1r12528 results DELETE

[Ethan Mallove]

When trying to DELETE some Test Runs for
1.2b1r12528 - I accidentally whacked 3,500 Test
Runs. If you have the mttdatabase_debug files
somewhere I can put them back in the database. My
apologies.

-- 
-Ethan

[MTT users] New MTT home page

[Ethan Mallove]

Folks,

The MTT home page has been updated to display a number of
quick links divided up by date range, organization, and
testing phase (MPI installation, test build, and test run).
The custom and summary reports are linked from there.

http://www.open-mpi.org/mtt/

-- 
-Ethan

Re: [MTT users] make distclean option?

[Ethan Mallove]

Here's a workaround that might work. Put something like this
your INI file:

[Test build: foo]
test_get = foo
module = Shell
shell_build_command = make

[Test build: foo clean]
test_get = foo
module = Shell
shell_build_command = make distclean

Then do:

$ client/mtt --no-section clean ...

In a subsequent mtt run, do:

$ client/mtt --section clean --force

Or similarly, put all your [build: clean] sections in a
seperate INI file, and after your initial "real" mtt run do
something like:

$ client/mtt clean.ini --force

-Ethan


On Mon, Nov/06/2006 01:15:24PM, Jeff Squyres wrote:
> The intent was to leave the source around so that you could go look  
> at things if there was a problem (especially during a build).
> 
> A probably completed "Trim" phase would have solved this. :-\ (e.g.,  
> keep_successful_mpi_builds = 0).  But Trim is unfortunately quite  
> complex to implement -- that's the only reason it hasn't been done yet.
> 
> As a short-term options, which would you prefer:
> 
> 1. a post_build option that will run after a success build/install
> 2. an option to simply whack the entire source tree after a  
> successful build/install
> 
> 
> On Nov 6, 2006, at 1:10 PM, Josh Hursey wrote:
> 
> > Is there a 'post_build' flag in the [MPI Install] section? I'd like
> > to be able to execute a script or 'make distclean' after a version of
> > a branch has been built and installed.
> >
> > The problem is that we are getting close to our quota on some of the
> > machines that we are using (every night we generate a directory about
> > 1.3 GB, and that quickly adds up). Since the 'source' directory is
> > not really useful (except for config.log) once the branch is
> > installed I'd like to shrink it a bit (or remove it all together).
> >
> > Is there any way to do this currently?
> >
> > Cheers,
> > Josh
> >
> > ___
> > mtt-users mailing list
> > mtt-us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Discussion on teleconf yesterday? (raw SQL)

[Ethan Mallove]

FYI - in the open-mpi.org/mtt, you can check the "SQL"
checkbox to see the raw SQL queries. Here's the "Odin" query
showing the SQL:

http://tinyurl.com/yd7oob

You can modify and play around with those on the live
database, e.g.:

$ echo "... some SQL ... " | psql -d mtt -U iu

I also like the pg_dump command. E.g., view the live schema:

$ pg_dump -s -d mtt -U iu

-Ethan


> > 
> > Basically our problems are:
> >   - Testing results come in at various times as they complete, we  
> > would really like a 'status report' at 8 am every day finished or not.
> 
> For now, could you load up this webpage everyday at 8am?
> 
> http://tinyurl.com/ydt777
> 
> 

Re: [MTT users] Discussion on teleconf yesterday?

[Ethan Mallove]

On Fri, Oct/27/2006 10:31:44AM, Josh Hursey wrote:
> 
> On Oct 27, 2006, at 7:39 AM, Jeff Squyres wrote:
> 
> > On Oct 25, 2006, at 10:37 AM, Josh Hursey wrote:
> >
> >> The discussion started with the bug characteristics of v1.2 versus
> >> the trunk.
> >
> > Gotcha.
> >
> >> It seemed from the call that IU was the only institution that can
> >> asses this via MTT as noone else spoke up. Since people were
> >> interested in seeing things that were breaking I suggested that I
> >> start forwarding the IU internal MTT reports (run nightly and
> >> weekly) to the test...@open-mpi.org. This was meet by Brain
> >> insisting that it would result in "thousands" of emails to the
> >> development list. I clarified that it is only 3 - 4 messages a day
> >> from IU. However if all other institutions do this then it would be
> >> a bunch of email (where 'a bunch' would still be less than
> >> 'thousands'). That's how we got to a 'we need a single summary
> >> presented to the group' comment. It should be noted that we brought
> >> up IU sending to the 'test...@open-mpi.org' list as a bandaid until
> >> MTT could do it better.
> >
> > How about sending them to me and Ethan?
> 
> Sure I can add you both to the list if you like.
> 
> >
> >> This single summary can be email or a webpage that people can
> >> check. Rich said that he would prefer a webpage, and noone else
> >> really had a comment. That got us talking about the current summary
> >> page that MTT generates. Tim M mentioned that the current website
> >> is difficult to figure out how to get the answers you need. I
> >> agree, it is hard [usability] for someone to go to the summary page
> >> and answer the question "So what failed from IU last night, and how
> >> does that differ from Yesterday -- e.g., what regressed and
> >> progressed yesterday at IU?". The website is flexible enough to due
> >> it, but having a couple of basic summary pages would be nice for
> >> basic users. What that should look like we can discuss further.
> >
> > Agreed; we aren't super-fond of the current web page, either.  Do you
> > guys want to have a teleconf to go over the current status of MTT,
> > where you want it to go, etc.?  I consider IU's input here quite
> > important, since you're the ones pushing the boundaries, flexing
> > MTT's muscles, etc.
> 
> In my previous email I suggested a couple of questions that I would  
> like a webpage to answer. A teleconf might be good to talk about some  
> of the various items that IU is trying to do around MTT.
> 
> >
> >> The IU group really likes the emails that we currently generate. A
> >> plain-text summary of the previous run. I posted copies on the MTT
> >> bug tracker here:
> >> http://svn.open-mpi.org/trac/mtt/ticket/61
> >> Currently we have not put the work in to aggregate the runs, so for
> >> each INI file that we run we get 1 email to the IU group. This is
> >> fine for the moment, but as we add the rest of the clusters and
> >> dimensions in the testing matrix we will need MTT to aggregate the
> >> results for us and generate such an email.
> >
> > Ok.
> >
> > We created another ticket yesterday to make a new MTT Reporter (our
> > internal plugins) that duplicates this output format.  It actually
> > shouldn't be that hard -- we don't have to do parsing to get the
> > numbers that you're reporting; we have access to the actual data.  So
> > it's mostly caching the data, calculating the totals that you're
> > calculating, and printing in your output format.
> >
> > Ethan has some other short tasks to do before he gets to this, but
> > its near the top of the priority list.  You can see the current
> > workflow on the wiki (this is a living document; it keeps changing as
> > requirements, etc. change):
> >
> >  http://svn.open-mpi.org/trac/mtt/wiki/TaskPlan
> >
> 
> Awesome Thanks! :)
> 
> >> So I think the general feel of the discussion is that we need the
> >> following from MTT:
> >>  - A 'basic' summary page providing answers to some general
> >> frequently asked queries. The current interface is too advanced for
> >> the current users.
> >
> > We have the summary.php page, but I personally have never found it
> > too useful.  :-)
> >
> > We're getting towards a full revamp of reporter.php (got some other
> > tasks to complete first, but we're definitely starting to think about
> > it) -- got any ideas / input?  Our "haven't thought about it much
> > yet" idea is to be more menu/Q-A driven with a few common queries
> > easily available (rather than a huge, complicated single screen).
> 
> See previous email for some general ideas. Tim M might have a few  
> more that he would like to see since he is the one at IU that is  
> watching the nightly results the closest.
> 
> >
> >>  - A summary email [in plain-text preferably] similar to the one
> >> that IU generated showing an aggregation of the previous nights
> >> results for (a) all reporters (b) my institution [so I can track
> >> them down and file bugs].
> 

Re: [MTT users] Bproc support in MTT

[Ethan Mallove]

On Wed, Oct/18/2006 09:28:01AM, James W. Barker wrote:
> Ethan,
> 
> In an email exchange with Jeff Squyres regarding setting the number 
> of processors on a Bproc system the following statement was made:
> 
> Are you running under bproc or some other scheduler?  Right now, MTT 
> understands SLURM, Torque, and LoadLeveler.  If you're running under 
> Bproc, we can easily add support for it into MTT (I'll need some 
> information from you since I don't have any BProc systems myself).
> 
> Jeff went on to say:
> 
> Ethan Mallove is the engineer at Sun who is heading up most of the 
> MTT efforts these days; the mails go to all of us, so he can reply 
> even when I'm unavailable.
> 
> So I am asking what information do you need from me to add Bproc 
> support to MTT?
>

Are NODES and NODELIST the only env vars that BProc uses (I
gleaned this info from
http://asci-training.lanl.gov/BProc/)?

MTT is set up to detect environment variables and hostfiles
used by Slurm, PBS, N1GE, and LoadLeveler and set a max
value for -np based on those env vars.  While we add the
smarts to MTT to do that for BProc, you could also hardcode
the -np value (e.g., 2) in your ini's "MPI Details" section
like so:

exec = mpirun @hosts@ -np 2 --prefix _prefix() _executable() 
_argv()

Thanks,
Ethan


> Thanks,
> Jim Barker
> 
> James W. Barker,  Ph.D.
> Los Alamos National Laboratory
> Computer, Computational and Statistical Sciences Division
> Advanced Computing Laboratory - Application Communications and 
> Performance Research Team
> 505-665-9558 
> 
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] MTT reporter module settings

[Ethan Mallove]

Hi Jim,

On Wed, Oct/18/2006 12:02:24PM, James W. Barker wrote:
> Ethan,
> 
> Do you know the correct values for the elements of the reporter 
> module of the MTT .ini file?  These are the values I currently have 
> set in my .ini file.
> 
> [Reporter: IU database]
> module = MTTDatabase
> 
> mttdatabase_realm = OMPI
> mttdatabase_url = https://www.open-mpi.org/mtt/submit/


Do you access the web through a proxy server? If so, you
need to set your http_proxy/https_proxy env var(s) to that
proxy.


> # OMPI Core: Change this to be the username and password for your
> # submit user.  Get this from the OMPI MTT administrator.
> mttdatabase_username = lanl
> mttdatabase_password = XXX


I take it you *do* know your password, but just didn't email
it to the list.


> # OMPI Core: Change this to be some short string identifying your
> # cluster.
> mttdatabase_platform = odin

This won't effect your connectivity with the mtt database,
but you probably want to do something like:

mttdatabase_platform = lanl

-Ethan

> 
> MTT reported it could not ping https://www.open-mpi.org/mtt/submit/, 
> what is the correct URL?  Is the mttdatabase_realm = OMPI?  Is the 
> mttdatabase_platform = odin?
> 
> Thanks,
> Jim Barker
> 
> 
> James W. Barker,  Ph.D.
> Los Alamos National Laboratory
> Computer, Computational and Statistical Sciences Division
> Advanced Computing Laboratory - Application Communications and 
> Performance Research Team
> 505-665-9558 
> 
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Tests build, but do not run

[Ethan Mallove]

On Thu, Oct/12/2006 03:01:53PM, Jeff Squyres wrote:
> On Oct 11, 2006, at 11:34 AM, Ethan Mallove wrote:
> 
> >So if I do put the four combinations of platform/bitness in
> >a single ini file, I then have to do some ugly ini param
> >overriding to line up the sections, e.g.,:
> >
> >Command 1)
> >
> >$ cat /home/em162155/mtt/all.ini |
> > client/mtt  -p -d -
> >  [...]
> >   mpi_get='ompi-nightly-trunk'
> >   "intel:test_build='intel $arch $bit'"
> >   "imb:test_build='imb $arch $bit'"
> >   "ibm:test_build='ibm $arch $bit'"
> >   "trivial:test_build='trivial $arch $bit'"
> >
> >I was thinking it would be nice if I could do something
> >like this in my ini:
> >
> >[Test get: intel all]
> >suite_name = intel
> >[Test build: intel sparc 32]
> >suite_name = intel
> >[Test build: intel sparc 64]
> >suite_name = intel
> >[Test build: intel i386 32]
> >suite_name = intel
> >[Test build: intel i386 64]
> >suite_name = intel
> >[Test run: intel all]
> >suite_name = intel
> 
> Why do you need different test build sections?
>

Different intel_ompi_tests_(f|c)flags parameters for each.

-Ethan


> -- 
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems
> 

Re: [MTT users] Post run result submission

[Ethan Mallove]

I've posted the helper script here:

http://svn.open-mpi.org/trac/mtt/wiki/SubmitHelper

Let me know how it works.

-Ethan


On Tue, Sep/26/2006 04:06:01PM, Jeff Squyres wrote:
> For the moment, that might be sufficient.
> 
> What HLRS does is open ssh tunnels back to the head node and then http put's
> through those back to the IU database.  Icky, but it works.
> 
> The problem is that there are some other higher-priority items that we need
> to get done in MTT (performane measurements, for example) that, since there
> are [icky] workarounds for http puts, we put the whole "disconnected
> scenario" stuff at a lower priority.  :-(
> 
> 
> On 9/26/06 3:51 PM, "Ethan Mallove" <ethan.mall...@sun.com> wrote:
> 
> > I have an unpretty solution that maybe could serve as a
> > stopgap between now and when we implement the "disconnected
> > scenarios" feature.  I have a very simple and easy-to-use
> > perl script that just HTTP POSTs a debug file (what *would*
> > have gone to the database). E.g.,
> > 
> > $ ./poster.pl -f 'mttdatabase_debug*.txt'
> > 
> > (Where mttdatabase_debug would be what you supply to the
> > mttdatabase_debug_filename ini param in the "IU Database"
> > section.)
> > 
> > I think this would fill in your missing * step below.
> > 
> > Does that sound okay, Jeff?
> > 
> > -Ethan
> > 
> > 
> > On Tue, Sep/26/2006 03:25:08PM, Josh Hursey wrote:
> >> So the login node is the only one that has a window to the outside
> >> world. I can't access the outside world from within an allocation.
> >> 
> >> So my script does:
> >>   - Login Node:
> >> 1) Get MPI Tarballs
> >>   - 1 Compute node:
> >> 0) Allocate a compute node to compile.
> >> 1) Build/Install MPI builds
> >> 2) Deallocate compute node
> >>   - Login Node:
> >> 1) Get MPI Test sources
> >>   - N Compute Nodes:
> >> 0) Allocate N compute Nodes to run the tests on
> >> 1) Build/Install Tests
> >> 2) Run the tests...
> >>   - Login Node:
> >> 0) Check to make sure we are all done (scheduler didn't kill the
> >>job, etc.).
> >> 1) Report results to MTT *
> >> 
> >> * This is what I am missing currently.
> >> 
> >> I currently have the "Reporter: IU Database" section commented out so
> >> that once the tests finish they don't try to post the database, since
> >> they can't see the outside world.
> >> 
> >> On Sep 26, 2006, at 3:17 PM, Ethan Mallove wrote:
> >> 
> >>> On Tue, Sep/26/2006 02:01:41PM, Josh Hursey wrote:
> >>>> I'm setting up MTT on BigRed at IU, and due to some visibility
> >>>> requirements of the compute nodes I segment the MTT operations.
> >>>> Currently I have a perl script that does all the svn and wget
> >>>> interactions from the login node, then compiles and runs on the
> >>>> compute nodes. This all seems to work fine.
> >>>> 
> >>>> Now I am wondering how to get the textfile results that were
> >>>> generated back to the MTT database once the run has finished.
> >>>> 
> >>> 
> >>> If you run the "MPI Install", "Test build", and "Test run"
> >>> sections from the same machine (call it the
> >>> "Install-Build-Run" node), I would think you could then
> >>> additionaly run the "Reporter: IU Database" section. Or can
> >>> you not do the HTTP POST from Install-Build-Run node?
> >>> 
> >>> -Ethan
> >>> 
> >>>> I know HLRS deals with this situation, is there a supported way of
> >>>> doing this yet or is it a future work item still?
> >>>> 
> >>>> Currently I have a method to send a summary email to our team after
> >>>> the results are generated, so this isn't a show stopper for IU or
> >>>> anything, just something so we can share our results with the rest of
> >>>> the team.
> >>>> 
> >>>> Cheers,
> >>>> Josh
> >>>> ___
> >>>> mtt-users mailing list
> >>>> mtt-us...@open-mpi.org
> >>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >> 
> >> ___
> >> mtt-users mailing list
> >> mtt-us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> 
> 
> -- 
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems

Re: [MTT users] Post run result submission

[Ethan Mallove]

On Tue, Sep/26/2006 02:01:41PM, Josh Hursey wrote:
> I'm setting up MTT on BigRed at IU, and due to some visibility  
> requirements of the compute nodes I segment the MTT operations.  
> Currently I have a perl script that does all the svn and wget  
> interactions from the login node, then compiles and runs on the  
> compute nodes. This all seems to work fine.
> 
> Now I am wondering how to get the textfile results that were  
> generated back to the MTT database once the run has finished.
> 

If you run the "MPI Install", "Test build", and "Test run"
sections from the same machine (call it the
"Install-Build-Run" node), I would think you could then
additionaly run the "Reporter: IU Database" section. Or can
you not do the HTTP POST from Install-Build-Run node? 

-Ethan

> I know HLRS deals with this situation, is there a supported way of  
> doing this yet or is it a future work item still?
> 
> Currently I have a method to send a summary email to our team after  
> the results are generated, so this isn't a show stopper for IU or  
> anything, just something so we can share our results with the rest of  
> the team.
> 
> Cheers,
> Josh
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] Timeouts for specific tests

[Ethan Mallove]

Not sure if this will work, but here goes (correct me if I'm
wrong, Jeff).

Try appending the following to the intel section of your ini (put
the names of the tests you expect to take 5 min, in a file named
'supposed_to_take_5_min' next to your other test lists):

simple_timeout:tests = _executables("src/" . 
"("supposed_to_take_5_min")")
simple_timeout:np = _max_procs()
simple_timeout:timeout = 300
simple_timeout:exclusive = 1

-Ethan

On Wed, Sep/20/2006 01:04:55PM, Josh Hursey wrote:
> Is it possible to specify a timeout for a specific test in a test  
> suite in addition to the test suite as a whole?
> 
> For example some of the Intel tests take about 6 minutes to complete  
> normally, but the rest of the tests usually finish in a minute or so.  
> I'd like to keep the 1 min tests from waiting 6 min when they fail.
> 
> Just trying to tune our runs a bit here at IU. Not a bit deal if it  
> can't at the moment.
> 
> Cheers,
> Josh
> 
> ___
> mtt-users mailing list
> mtt-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT users] perfbase.php not slurping test run data

[Ethan Mallove]

FYI ...

We've narrowed this issue down to all results with occurrence="once" will input,
but if occurrence="multiple" - we see #no_data#.

I posted a question to users@perfbase if you're interested in following it 
there:

http://perfbase.tigris.org/servlets/ReadMsg?list=users=126

-Ethan


Ethan Mallove wrote On 07/19/06 13:25,:
> Jeff Squyres (jsquyres) wrote On 07/18/06 19:43,:
> 
>>I would try this manually, without the php script in between.
>>Specifically:
>>
>>1. I would modify the .php script to write out a file
>>2. send a submit through the .php file
>>3. examine the resulting file and ensure that all the fields match up to
>>what you think they should be
>>4. manually submit this file to perfbase and see what happens
>>
> 
> 
> I can't get it to work outside of the php script either:
> 
> $ date
> Wed Jul 19 12:34:49 EDT 2006
> $ cat pb.in | perfbase input -v -i -d www/inp_test_run_correctness.xml -
> #* importing data from
> - (parsed according to www/inp_test_run_correctness.xml)
> #* Parsing input description from XML file www/inp_test_run_correctness.xml
> #* Processing data file  -
> #+ Creating new run with index 5
> $ cat pb.in
> hostname: burl-ct-v20z-0
> os_name: SunOS
> os_version: SunOS 5.10
> platform_hardware: i86pc
> platform_type: solaris-10
> submit_timestamp: Mon Jul 17 18:30:48 2006
> mpi_install_section_name: solaris i386
> mpi_name: ompi-nightly-v1.0
> mpi_version: 1.0.3a1r10793
> mtt_version_major: 0
> mtt_version_minor: 5
> perfbase_xml: inp_test_run_correctness.xml
> phase: Test run
> platform_id: IU Odin cluster
> start_timestamp: Mon Jul 17 18:30:45 2006
> stop_timestamp: Mon Jul 17 18:30:48 2006
> test_build_section_name: trivial
> test_command: mpirun -np 2 --prefix
> /workspace/em162155/hpc/mtt/trunk/installs/ompi-nightly-v1.0/solaris_i386/install
> ./f77_hello
> test_message: Passed
> test_name: f77_hello
> test_np: 2
> test_pass: 1
> test_run_section_name: trivial
> merge_stdout_stderr: 0
> mpi_get_section_name: solaris i386
> ENVIRONMENT_BEGIN
> foo = bar
> ENVIRONMENT_END
> STDERR_BEGIN
> blah blah blah
> STDERR_END
> STDOUT_BEGIN
> blah blah blah
> STDOUT_END
> $ perfbase ls -l --exp=test_run_correctness -v
> --show=mtt_version_major,mtt_version_minor,hostname,os_name,os_version,platform_hardware,platform_type,platform_id,environment,mpi_name,mpi_version,mpi_get_section_name,mpi_install_section_name,test_build_section_name,test_run_section_name,merge_stdout_stderr,test_command,test_message,test_name,test_np,test_pass,stderr,stdout,submit_timestamp,start_timestamp,stop_timestamp
> --nvals=999 | tail -2
> 5   postgres2006-07-19 12:34:49.00  0   5   burl-ct-v20z-0
> SunOS   SunOS 5.10  i86pc   solaris-10  IU Odin cluster foo = bar
> ompi-nightly-v1.0   1.0.3a1r10793   solaris i386solaris i386
> trivial trivial 0   #no_data#   #no_data#   #no_data#   #no_data#
> #no_data#   #no_data#   #no_data#   2006-07-17 18:30:48.00
> #no_data#   #no_data#
> 
> ---
> 
> The timestamp for the perfbase input and ls matches, which shows *something* 
> was
> inserted into perfbase, but shouldn't I see "Passed", "f77_hello", "2", etc.
> where the perfbase ls output shows #no_data#? (Note: I took the -u option out 
> of
> perfbase input to be more "strict" for the time being.)
> 
> I'm not able to insert data into one of the problematic fields with perfbase's
> fixed content option (-f name=val). E.g.,
> 
> $ cat pb.in | perfbase input -f test_name=f77_hello -v -i -d
> www/inp_test_run_correctness.xml -
> 
> Tried to delete the experiment and do a re-setup, but that didn't help.
> 
> $ perfbase delete --exp=test_run_correctness
> $ perfbase setup --desc=www/exp_test_run_correctness.xml
> 
> The pattern I see is that the problematic fields are set to
> occurrence="multiple". For these fields I've set retrigger=yes (which should 
> be
> the default) to force an update to occur if the row exists - no luck.
> 
> I'm not sure if this is a problem in the exp_*.xml files or the inp_*.xml 
> files?.
> 
> Also, how was it determined in the exp*.xml file, which fields were to be a
> "parameter" and which were to be a "result"? Looks like the only difference 
> is a
> parameter can nest 0 or more "valid" elements.
> 
> -Ethan
> 
> 
> 
>>That's what I would do.
>> 
>>
>>
>>
>>>-Original Message-
>>>From: mtt-users-boun...@open-mpi.org 
>>>[mailto:mtt-users-boun...@open-mpi.org] On Behalf Of Ethan Mallove
>&

Re: [MTT users] perfbase.php not slurping test run data

[Ethan Mallove]

Jeff Squyres (jsquyres) wrote On 07/18/06 19:43,:
> I would try this manually, without the php script in between.
> Specifically:
> 
> 1. I would modify the .php script to write out a file
> 2. send a submit through the .php file
> 3. examine the resulting file and ensure that all the fields match up to
> what you think they should be
> 4. manually submit this file to perfbase and see what happens
> 

I can't get it to work outside of the php script either:

$ date
Wed Jul 19 12:34:49 EDT 2006
$ cat pb.in | perfbase input -v -i -d www/inp_test_run_correctness.xml -
#* importing data from
- (parsed according to www/inp_test_run_correctness.xml)
#* Parsing input description from XML file www/inp_test_run_correctness.xml
#* Processing data file  -
#+ Creating new run with index 5
$ cat pb.in
hostname: burl-ct-v20z-0
os_name: SunOS
os_version: SunOS 5.10
platform_hardware: i86pc
platform_type: solaris-10
submit_timestamp: Mon Jul 17 18:30:48 2006
mpi_install_section_name: solaris i386
mpi_name: ompi-nightly-v1.0
mpi_version: 1.0.3a1r10793
mtt_version_major: 0
mtt_version_minor: 5
perfbase_xml: inp_test_run_correctness.xml
phase: Test run
platform_id: IU Odin cluster
start_timestamp: Mon Jul 17 18:30:45 2006
stop_timestamp: Mon Jul 17 18:30:48 2006
test_build_section_name: trivial
test_command: mpirun -np 2 --prefix
/workspace/em162155/hpc/mtt/trunk/installs/ompi-nightly-v1.0/solaris_i386/install
./f77_hello
test_message: Passed
test_name: f77_hello
test_np: 2
test_pass: 1
test_run_section_name: trivial
merge_stdout_stderr: 0
mpi_get_section_name: solaris i386
ENVIRONMENT_BEGIN
foo = bar
ENVIRONMENT_END
STDERR_BEGIN
blah blah blah
STDERR_END
STDOUT_BEGIN
blah blah blah
STDOUT_END
$ perfbase ls -l --exp=test_run_correctness -v
--show=mtt_version_major,mtt_version_minor,hostname,os_name,os_version,platform_hardware,platform_type,platform_id,environment,mpi_name,mpi_version,mpi_get_section_name,mpi_install_section_name,test_build_section_name,test_run_section_name,merge_stdout_stderr,test_command,test_message,test_name,test_np,test_pass,stderr,stdout,submit_timestamp,start_timestamp,stop_timestamp
--nvals=999 | tail -2
5   postgres2006-07-19 12:34:49.00  0   5   burl-ct-v20z-0
SunOS   SunOS 5.10  i86pc   solaris-10  IU Odin cluster foo = bar
ompi-nightly-v1.0   1.0.3a1r10793   solaris i386solaris i386
trivial trivial 0   #no_data#   #no_data#   #no_data#   #no_data#
#no_data#   #no_data#   #no_data#   2006-07-17 18:30:48.00
#no_data#   #no_data#

---

The timestamp for the perfbase input and ls matches, which shows *something* was
inserted into perfbase, but shouldn't I see "Passed", "f77_hello", "2", etc.
where the perfbase ls output shows #no_data#? (Note: I took the -u option out of
perfbase input to be more "strict" for the time being.)

I'm not able to insert data into one of the problematic fields with perfbase's
fixed content option (-f name=val). E.g.,

$ cat pb.in | perfbase input -f test_name=f77_hello -v -i -d
www/inp_test_run_correctness.xml -

Tried to delete the experiment and do a re-setup, but that didn't help.

$ perfbase delete --exp=test_run_correctness
$ perfbase setup --desc=www/exp_test_run_correctness.xml

The pattern I see is that the problematic fields are set to
occurrence="multiple". For these fields I've set retrigger=yes (which should be
the default) to force an update to occur if the row exists - no luck.

I'm not sure if this is a problem in the exp_*.xml files or the inp_*.xml 
files?.

Also, how was it determined in the exp*.xml file, which fields were to be a
"parameter" and which were to be a "result"? Looks like the only difference is a
parameter can nest 0 or more "valid" elements.

-Ethan


> That's what I would do.
>  
> 
> 
>>-----Original Message-
>>From: mtt-users-boun...@open-mpi.org 
>>[mailto:mtt-users-boun...@open-mpi.org] On Behalf Of Ethan Mallove
>>Sent: Tuesday, July 18, 2006 6:33 PM
>>To: mtt-us...@open-mpi.org
>>Subject: [MTT users] perfbase.php not slurping test run data
>>
>>perfbase.php seems to only populate perfbase with mpi_install 
>>and test_build
>>data. In other words, data correlating to these files is 
>>getting input to
>>perfbase:
>>
>>  inp_mpi_install.xml
>>  inp_test_build.xml
>>
>>Data correlating to this file is _not_ getting input to perfbase:
>>
>>  inp_test_run_correctness.xml
>>
>>I verified this by looking at the perfbase data with pg_dump 
>>(postgres's
>>command-line tool to dump the entire database), which shows 
>>that there is no
>>data for these fields:
>>
>>test_pass
>>test_command
>>test_name
>>test_np
>>test_message
>>

Re: [MTT users] [MTT svn] svn:mtt r196 - in trunk: lib/MTT lib/MTT/Reporter server

[Ethan Mallove]

Here's the log which was not added to the repository because I accidentally
typed "c" at the "a)bort, c)ontinue, e)dit" svn commit prompt.

- Show user where the perfbase HTTP POST is going in host:port/dir
  format.
- env_proxy => 1 is unreliable. Look for the proxy in the user's env and
  explicitly set it for the UserAgent
- Allow for sending data to perfbase _and_ output to perfbase debug
  files (before it had to be one or the other). Note: to turn off
  uploading test results to perfbase, set perfbase_url to an invalid
  url
- Adjusted some MTT defaults
- Some typos fixed

-Ethan


em162...@osl.iu.edu wrote On 07/14/06 11:47,:
> Author: em162155
> Date: 2006-07-14 11:46:44 -0400 (Fri, 14 Jul 2006)
> New Revision: 196
> 
> Modified:
>trunk/lib/MTT/Defaults.pm
>trunk/lib/MTT/Reporter/Perfbase.pm
>trunk/server/mtt_config.php.perfbase.sample
>trunk/server/perfbase.php
> Log:
> 
> 
> Modified: trunk/lib/MTT/Defaults.pm
> ===
> --- trunk/lib/MTT/Defaults.pm 2006-07-11 16:42:27 UTC (rev 195)
> +++ trunk/lib/MTT/Defaults.pm 2006-07-14 15:46:44 UTC (rev 196)
> @@ -25,7 +25,7 @@
>  
>  http_agents => "wget lynx curl",
>  
> -known_compiler_names => "gnu pgi intel kai absoft pathscale none",
> +known_compiler_names => "gnu pgi intel kai absoft pathscale sun none",
>  };
>  
>  # User-defined configuration
> @@ -60,7 +60,7 @@
>  
>  # Test run phase
>  our $Test_run = {
> -perfbase_xml => "inp_test_run.xml",
> +perfbase_xml => "inp_test_run_correctness.xml",
>  argv => "",
>  np => 2,
>  np_ok => 1,
> 
> Modified: trunk/lib/MTT/Reporter/Perfbase.pm
> ===
> --- trunk/lib/MTT/Reporter/Perfbase.pm2006-07-11 16:42:27 UTC (rev 
> 195)
> +++ trunk/lib/MTT/Reporter/Perfbase.pm2006-07-14 15:46:44 UTC (rev 
> 196)
> @@ -3,6 +3,7 @@
>  # Copyright (c) 2005-2006 The Trustees of Indiana University.
>  # All rights reserved.
>  # Copyright (c) 2006  Cisco Systems, Inc.  All rights reserved.
> +# Copyright (c) 2006  Sun Microsystems, Inc.  All rights reserved.
>  # $COPYRIGHT$
>  # 
>  # Additional copyrights may follow
> @@ -25,6 +26,7 @@
>  my $password;
>  my $realm;
>  my $url;
> +my $port;
>  
>  # platform common name
>  my $platform;
> @@ -58,7 +60,7 @@
>  ++$count if ($password);
>  ++$count if ($realm);
>  if ($count > 0 && $count != 3) {
> -Warning("Perfbase Reporter section [$section]: if password, 
> username, or relam is specified, they all must be specified.\n");
> +Warning("Perfbase Reporter section [$section]: if password, 
> username, or realm is specified, they all must be specified.\n");
>  return undef;
>  }
>  $platform = Value($ini, $section, "perfbase_platform");
> @@ -66,24 +68,40 @@
>  # Extract the host and port from the URL.  Needed for the
>  # credentials section.
>  
> -my $port;
> +my $dir;
>  my $host = $url;
> -if ($host =~ /http:\/\/([-a-zA-Z0-9.]+):(\d+)/) {
> +if ($host =~ /(http:\/\/[-a-zA-Z0-9.]+):(\d+)\/(.*)$/) {
>  $host = $1;
>  $port = $2;
> -} elsif ($host =~ /http:\/\/([-a-zA-Z0-9.]+)/) {
> +$dir = $3;
> +} elsif ($host =~ /(http:\/\/[-a-zA-Z0-9.]+)\/(.*)$/) {
>  $host = $1;
> +$dir = $2;
>  $port = 80;
> -} elsif ($host =~ /https:\/\/([-a-zA-Z0-9.]+)/) {
> +} elsif ($host =~ /(https:\/\/[-a-zA-Z0-9.]+)\/(.*)$/) {
>  $host = $1;
> +$dir = $2;
>  $port = 443;
>  } else {
>  return undef;
>  }
> +$url = "$host:$port/$dir";
>  
> +# Look for the proxy server in the environment,
> +# and take the first one we find
> +my $proxy;
> +foreach my $p (grep(/http_proxy/i, keys %ENV)) {
> +if ($ENV{$p}) {
> +$proxy = $ENV{$p};
> +$proxy = "http://$proxy; if ($proxy !~ /https?:/);
> +last;
> +}
> +}
> +
>  # Create the Perl LWP stuff to setup for HTTP PUT's later
>  
> -$ua = LWP::UserAgent->new({ env_proxy => 1 });
> +$ua = LWP::UserAgent->new();
> +$ua->proxy(['http', 'ftp'], $proxy);
>  $ua->agent("MPI Test Perfbase Reporter");
>  if ($realm && $username && $password) {
>  Verbose("   Set HTTP credentials for realm \"$realm\"\n");
> @@ -186,10 +204,10 @@
>  PBXML => $xml,
>  };
>  
> -if (0 && !$debug_filename) {
> +if ($url) {
>  # Do the post and get the response.
>  
> -Debug("Submitting to perbase...\n");
> +Debug("Submitting to perfbase...\n");
>  my $response = $ua->post($url, $form);
>  if ($response->is_success()) {
>