Re: [MTT devel] mtt/server/gds

[Ralph Castain]

Sorry - I didn't look at it that carefully. I'd have to defer to Josh as I'm 
not sure how it relates to his CherryPy server. It looks like an alternative 
implementation from Voltaire 10 years ago.

I'm meeting with him in 30 min and can ask then, if he doesn't see this before 
then.


On Jul 25, 2019, at 11:14 AM, Rezanka, Deb via mtt-devel 
mailto:mtt-devel@lists.open-mpi.org> > wrote:

See the Subject line. mtt/server/gds
 --
Deb Rezanka 
B Schedule
HPC-ENV
Office 9, 2nd floor Research Park
TA-03, Building 4200, Room 203
Los Alamos National Laboratory
  From: mtt-devel mailto:mtt-devel-boun...@lists.open-mpi.org> > on behalf of Ralph Castain 
mailto:r...@open-mpi.org> >
Reply-To: Development list for the MPI Testing Tool 
mailto:mtt-devel@lists.open-mpi.org> >
Date: Thursday, July 25, 2019 at 12:13 PM
To: Development list for the MPI Testing Tool mailto:mtt-devel@lists.open-mpi.org> >
Subject: Re: [MTT devel] mtt/server/gds
 What "gds" directory are you talking about? Can you provide the full path to 
it?


On Jul 25, 2019, at 11:04 AM, Rezanka, Deb via mtt-devel 
mailto:mtt-devel@lists.open-mpi.org> > wrote:
 Does anyone know if the gds directory is being used at all? If not, could we 
get rid of it.
 --
Deb Rezanka 
B Schedule
HPC-ENV
Office 9, 2nd floor Research Park
TA-03, Building 4200, Room 203
Los Alamos National Laboratory
 ___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org <mailto:mtt-devel@lists.open-mpi.org> 
https://lists.open-mpi.org/mailman/listinfo/mtt-devel


___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org <mailto:mtt-devel@lists.open-mpi.org> 
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

Re: [MTT devel] mtt/server/gds

[Ralph Castain]

What "gds" directory are you talking about? Can you provide the full path to it?

On Jul 25, 2019, at 11:04 AM, Rezanka, Deb via mtt-devel 
mailto:mtt-devel@lists.open-mpi.org> > wrote:

Does anyone know if the gds directory is being used at all? If not, could we 
get rid of it.
 --
Deb Rezanka 
B Schedule
HPC-ENV
Office 9, 2nd floor Research Park
TA-03, Building 4200, Room 203
Los Alamos National Laboratory
 ___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org  
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

Re: [MTT devel] MTT Skype Meeting

[Barella, Richard T]

Try again

From: mtt-devel [mailto:mtt-devel-boun...@lists.open-mpi.org] On Behalf Of 
Rezanka, Deb via mtt-devel
Sent: Tuesday, November 6, 2018 11:27 AM
To: Van Dresser, Daniel N 
Cc: Rezanka, Deb ; mtt-devel@lists.open-mpi.org
Subject: [MTT devel] MTT Skype Meeting

Hi,

I have been trying to join the meeting today but could not seem to connect 
either through skype on the computer or over the phone.  I had some questions 
about the TestRun allocate_cmd, deallocate_cmd.  Using command = mpirun in 
[LauncherDefaults] does not do the test runs in the slurm allocation. Can 
anyone help?

Deb Rezanka
___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

Re: [MTT devel] MTT docs now online

[Peter Gottesman (pgottesm) via mtt-devel]

Awesome. Thanks Akshaya!
Am I understanding correctly that Travis is deploying from the docs/ directory?

From: mtt-devel  on behalf of 
"r...@open-mpi.org" 
Reply-To: Development list for the MPI Testing Tool 

Date: Thursday, July 12, 2018 at 9:04 AM
To: Development list for the MPI Testing Tool , 
MTT Users 
Subject: [MTT devel] MTT docs now online

We finally got the bugs out and the documentation for Python MTT implementation 
is now online at https://open-mpi.github.io/mtt.
It also picked up the Perl stuff, but we’ll just ignore that little detail :-)

Thanks to Akshaya Jagannadharao for all the hard work to make that happen!

Ralph

___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

Ok, just got in to Chicago from my flight and am back online.

Mike: you are still not providing very much information.  :-\

Your first mails make it seem like MTT is continuing to run, but leaving 
"launchers" (assumedly mpirun processes) still running, but they have no 
children.  Which would be very weird for mpirun to do, if it has no children 
left.  This could be both an MTT and an ORTE bug, in this case.

But your last mail seems to imply that MTT is hanging indefinitely.

Can you please provide a clear, precise description of what is happening?

FWIW: Yes, we are killing the parent first now, to give mpirun a chance to 
cleanup / tell remote orteds to die / kill children processes / etc.  Killing 
the children first both doesn't test the common case of how people kill MPI 
processes (i.e., they kill mpirun), and it also doesn't allow mpirun to tell 
remote processes to die.

Do you run with --verbose output?  MTT should output messages like "*** Killing 
mpirun with SIGTERM", and the like.  Do you see timeout messages at all?  I.e., 
is MTT not entering the timeout code at all?

...etc.



On Jun 23, 2014, at 12:16 PM, Dave Goodell (dgoodell)  
wrote:

> On Jun 23, 2014, at 8:48 AM, Mike Dubman  wrote:
> 
>> btw, i think now, when parent process is killed before child, OS makes child 
>> as "" which stick around for good.
> 
> The grandparent should inherit the child.  If the grandparent then does not 
> wait(2) on the child, then the child will remain a zombie / defunct.  So in 
> our specific case, this behavior will depend on what the parent process of 
> mpirun is and whether it is waiting on child processes appropriately.
> 
> -Dave
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/mtt-devel/2014/06/0633.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Dave Goodell (dgoodell)]

On Jun 23, 2014, at 8:48 AM, Mike Dubman  wrote:

> btw, i think now, when parent process is killed before child, OS makes child 
> as "" which stick around for good.

The grandparent should inherit the child.  If the grandparent then does not 
wait(2) on the child, then the child will remain a zombie / defunct.  So in our 
specific case, this behavior will depend on what the parent process of mpirun 
is and whether it is waiting on child processes appropriately.

-Dave

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Mike Dubman]

it seems that mpirun got no signal (no evidence in the log). mtt was
spinning and mpirun was a only process who left on the node.
It was unclear why mtt did not kill mpirun.
will try to extract perl stacktrace from mtt on tomorrow`s nightly run.


On Mon, Jun 23, 2014 at 2:59 PM, Jeff Squyres (jsquyres)  wrote:

> On Jun 23, 2014, at 7:47 AM, Mike Dubman  wrote:
>
> > after patch, it killed child processes but kept mpirun ... itself.
>
> What does that mean -- are you saying that mpirun is still running?  Was
> mpirun sent a signal at all?  What kind of messages are being displayed?
>  ...etc.
>
> The commits fix important bugs for me and others.  Clearly, there's still
> something not right.  And of course I'm willing to track it down.  But I
> can't help you if you just say "it doesn't work."
>
> > before that patch - all processes were killed (and you are right,
> "mpirun died right at the end of the timeout" was reported)
>
> ...which led to many months of misleading ORTE debugging, BTW.  :-\
>  That's why this commit was introduced into MTT -- in the quest of finally
> fixing both the mysterious ORTE hangs and the erroneous timeouts/"mpirun
> died right at the end" messages.
>
> > but at least it left the cluster in the clean state w/o leftovers.
> > now many "orphan" launchers  are alive from previous invocations.
>
> Does "launchers" = mpirun?
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> Link to this post:
> http://www.open-mpi.org/community/lists/mtt-devel/2014/06/0629.php
>

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

On Jun 23, 2014, at 7:47 AM, Mike Dubman  wrote:

> after patch, it killed child processes but kept mpirun ... itself.

What does that mean -- are you saying that mpirun is still running?  Was mpirun 
sent a signal at all?  What kind of messages are being displayed?  ...etc.

The commits fix important bugs for me and others.  Clearly, there's still 
something not right.  And of course I'm willing to track it down.  But I can't 
help you if you just say "it doesn't work."

> before that patch - all processes were killed (and you are right, "mpirun 
> died right at the end of the timeout" was reported)

...which led to many months of misleading ORTE debugging, BTW.  :-\  That's why 
this commit was introduced into MTT -- in the quest of finally fixing both the 
mysterious ORTE hangs and the erroneous timeouts/"mpirun died right at the end" 
messages.

> but at least it left the cluster in the clean state w/o leftovers.
> now many "orphan" launchers  are alive from previous invocations.

Does "launchers" = mpirun?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Mike Dubman]

after patch, it killed child processes but kept mpirun ... itself.

before that patch - all processes were killed (and you are right, "mpirun
died right at the end of the timeout" was reported) but at least it left
the cluster in the clean state w/o leftovers.
now many "orphan" launchers  are alive from previous invocations.


On Mon, Jun 23, 2014 at 2:18 PM, Jeff Squyres (jsquyres)  wrote:

> There was actually quite a bit of testing before this was committed.  This
> commit resolved a lot of hangs across multiple organizations.
>
> Can you be more specific as to what is happening?
>
> The prior code was killing child processes before mpirun itself, for
> example, which has led MTT to erroneously report that mpirun died right at
> the end of the timeout without being killed.  This has been ongoing for
> many months, at a minimum.
>
>
>
>
> On Jun 23, 2014, at 4:37 AM, Mike Dubman  wrote:
>
> > this commit does more harm then good.
> > we experience following:
> >
> > - some child processes still running after timeout and mtt killed the
> job.
> >
> > before this commit - it worked fine.
> > please revert and test more.
> >
> >
> >
> > On Sat, Jun 21, 2014 at 3:30 PM, MPI Team  wrote:
> > The branch, master has been updated
> >via  016088f2a0831b32ab5fd6f60f4cabe67e92e594 (commit)
> >via  7fb4c6a4c9d71be127ea53bd463178510577f71f (commit)
> >via  381ba177d835a54c3197d846f5a4edfc314efe27 (commit)
> >via  cfdd29de2012eeb7592706f00dd07a52dd48cf6b (commit)
> >via  940030ca20eb1eaf256e898b83866c1cb83aca5c (commit)
> >   from  c99ed7c7b159a2cab58a251bd7c0dad8972ff901 (commit)
> >
> > Those revisions listed above that are new to this repository have
> > not appeared on any other notification email; so we list those
> > revisions in full, below.
> >
> > - Log -
> >
> https://github.com/open-mpi/mtt/commit/016088f2a0831b32ab5fd6f60f4cabe67e92e594
> >
> > commit 016088f2a0831b32ab5fd6f60f4cabe67e92e594
> > Author: Jeff Squyres 
> > Date:   Sat Jun 21 04:58:45 2014 -0700
> >
> > DoCommand: several fixes to kill_proc logic
> >
> > 1. Fix the kill(0, $pid) test to see if the process was still alive.
> >
> > 2. Rename _kill_proc() to _kill_proc_tree() to indicate that it's
> > really killing not only the PID in question, but also all of its
> > descendants.
> >
> > 3. In _kill_proc_tree(), change the order to kill the main PID first,
> > and ''then'' kill all the descendants.
> >
> > The main use case is when killing mpirun: if we kill mpirun's
> > descendants ''first'', mpirun will detect its childrens' deaths and
> > then cleanup and exit.  Later, when MTT finally gets around to
> killing
> > mpirun, MTT will detect that mpirun is already dead and therefore
> emit
> > a confusing "mpirun died right at end of timeout" message.  This is
> > misleading at best; it doesn't indicate what actually happened.
> >
> > However, if we kill mpirun first, it will take care of killing all of
> > its descendants.  MTT will therefore emit the right messages about
> > killing mpirun.  MTT will then redundantly try to kill a bunch of
> > now-nonexistent descendant processes of mpirun, but that's ok/safe.
> > We actually ''want'' this try-to-kill-mpirun's-descendants behavior
> to
> > handle the case when mpirun is misbehaving / not cleaning up its
> > descendants.
> >
> > 4. DoCommand() is used for more than launching mpirun, so pass down
> > $argv0 so that we can print the actual command name that is being
> > killed in various Verbose/Debug messages, not the hard-coded "mpirun"
> > string (which, in practice, was probably almost always correct, but
> > still...).
> > ---
> >  lib/MTT/DoCommand.pm | 78
> 
> >  1 file changed, 55 insertions(+), 23 deletions(-)
> >
> > diff --git a/lib/MTT/DoCommand.pm b/lib/MTT/DoCommand.pm
> > index 02cdb94..646ca31 100644
> > --- a/lib/MTT/DoCommand.pm
> > +++ b/lib/MTT/DoCommand.pm
> > @@ -2,7 +2,7 @@
> >  #
> >  # Copyright (c) 2005-2006 The Trustees of Indiana University.
> >  # All rights reserved.
> > -# Copyright (c) 2006-2013 Cisco Systems, Inc.  All rights reserved.
> > +# Copyright (c) 2006-2014 Cisco Systems, Inc.  All rights reserved.
> >  # Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
> >  # Copyright (c) 2007-2012 High Performance Computing Center Stuttgart,
> >  # University of Stuttgart.  All rights reserved.
> > @@ -57,23 +57,27 @@ sub DoCommand {
> >  ($time_arg, $no_execute) = @_;
> >  }
> >
> > +# This function is called for killing both mpirun and each of its
> > +# descendants.  We really only want to see verbose output for when we
> > +# kill mpirun itself, so 

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

There was actually quite a bit of testing before this was committed.  This 
commit resolved a lot of hangs across multiple organizations.

Can you be more specific as to what is happening?

The prior code was killing child processes before mpirun itself, for example, 
which has led MTT to erroneously report that mpirun died right at the end of 
the timeout without being killed.  This has been ongoing for many months, at a 
minimum.




On Jun 23, 2014, at 4:37 AM, Mike Dubman  wrote:

> this commit does more harm then good.
> we experience following:
> 
> - some child processes still running after timeout and mtt killed the job.
> 
> before this commit - it worked fine.
> please revert and test more.
>  
> 
> 
> On Sat, Jun 21, 2014 at 3:30 PM, MPI Team  wrote:
> The branch, master has been updated
>via  016088f2a0831b32ab5fd6f60f4cabe67e92e594 (commit)
>via  7fb4c6a4c9d71be127ea53bd463178510577f71f (commit)
>via  381ba177d835a54c3197d846f5a4edfc314efe27 (commit)
>via  cfdd29de2012eeb7592706f00dd07a52dd48cf6b (commit)
>via  940030ca20eb1eaf256e898b83866c1cb83aca5c (commit)
>   from  c99ed7c7b159a2cab58a251bd7c0dad8972ff901 (commit)
> 
> Those revisions listed above that are new to this repository have
> not appeared on any other notification email; so we list those
> revisions in full, below.
> 
> - Log -
> https://github.com/open-mpi/mtt/commit/016088f2a0831b32ab5fd6f60f4cabe67e92e594
> 
> commit 016088f2a0831b32ab5fd6f60f4cabe67e92e594
> Author: Jeff Squyres 
> Date:   Sat Jun 21 04:58:45 2014 -0700
> 
> DoCommand: several fixes to kill_proc logic
> 
> 1. Fix the kill(0, $pid) test to see if the process was still alive.
> 
> 2. Rename _kill_proc() to _kill_proc_tree() to indicate that it's
> really killing not only the PID in question, but also all of its
> descendants.
> 
> 3. In _kill_proc_tree(), change the order to kill the main PID first,
> and ''then'' kill all the descendants.
> 
> The main use case is when killing mpirun: if we kill mpirun's
> descendants ''first'', mpirun will detect its childrens' deaths and
> then cleanup and exit.  Later, when MTT finally gets around to killing
> mpirun, MTT will detect that mpirun is already dead and therefore emit
> a confusing "mpirun died right at end of timeout" message.  This is
> misleading at best; it doesn't indicate what actually happened.
> 
> However, if we kill mpirun first, it will take care of killing all of
> its descendants.  MTT will therefore emit the right messages about
> killing mpirun.  MTT will then redundantly try to kill a bunch of
> now-nonexistent descendant processes of mpirun, but that's ok/safe.
> We actually ''want'' this try-to-kill-mpirun's-descendants behavior to
> handle the case when mpirun is misbehaving / not cleaning up its
> descendants.
> 
> 4. DoCommand() is used for more than launching mpirun, so pass down
> $argv0 so that we can print the actual command name that is being
> killed in various Verbose/Debug messages, not the hard-coded "mpirun"
> string (which, in practice, was probably almost always correct, but
> still...).
> ---
>  lib/MTT/DoCommand.pm | 78 
> 
>  1 file changed, 55 insertions(+), 23 deletions(-)
> 
> diff --git a/lib/MTT/DoCommand.pm b/lib/MTT/DoCommand.pm
> index 02cdb94..646ca31 100644
> --- a/lib/MTT/DoCommand.pm
> +++ b/lib/MTT/DoCommand.pm
> @@ -2,7 +2,7 @@
>  #
>  # Copyright (c) 2005-2006 The Trustees of Indiana University.
>  # All rights reserved.
> -# Copyright (c) 2006-2013 Cisco Systems, Inc.  All rights reserved.
> +# Copyright (c) 2006-2014 Cisco Systems, Inc.  All rights reserved.
>  # Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
>  # Copyright (c) 2007-2012 High Performance Computing Center Stuttgart,
>  # University of Stuttgart.  All rights reserved.
> @@ -57,23 +57,27 @@ sub DoCommand {
>  ($time_arg, $no_execute) = @_;
>  }
> 
> +# This function is called for killing both mpirun and each of its
> +# descendants.  We really only want to see verbose output for when we
> +# kill mpirun itself, so only show output when the caller provides us
> +# with a $argv0 value.
>  sub _kill_proc_one {
> -my ($pid) = @_;
> +my ($pid, $argv0) = @_;
> 
>  # How long to wait after each kill()
>  my $wait_time = 5;
> 
>  # See if the proc is alive first
> -my $kid;
> -kill(0, $pid);
> -$kid = waitpid($pid, WNOHANG);
> -return "mpirun died right at end of timeout (MTT did not have to kill 
> it)"
> -if (-1 == $kid);
> +my $num_alive = kill(0, $pid);
> +return "$argv0 died right at end of timeout (MTT did not have to kill 
> it)"
> +if (0 == 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

On Apr 7, 2014, at 6:39 PM, Mike Dubman  wrote:

> somehow we run it with both, --verbose not enough to understand the problem 
> and --debug is too much.
> 
> maybe --trace is here to rescue?

It won't really solve the problem; it'll just create another level of argument 
about where a given message should go (now there will be 3 levels instead of 
2... eventually there will be 4, and then 5, and then ...).

Do you really need to run with --debug all the time?  I.e., do you have so many 
problems with MTT itself that you need to run with --debug?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Mike Dubman]

somehow we run it with both, --verbose not enough to understand the problem
and --debug is too much.

maybe --trace is here to rescue?



On Tue, Apr 8, 2014 at 1:36 AM, Jeff Squyres (jsquyres)
wrote:

> Yes.
>
> The intent is that --debug is *very* verbose, and is generally only
> useful when something goes wrong.
>
> I run Cisco's automated MTT with only --verbose.
>
>
>
> On Apr 7, 2014, at 6:35 PM, Mike Dubman  wrote:
>
> > ohh.. it is just flooding the log with same data for every test launch.
> >
> > maybe we should have verbose level in mtt?
> >
> >
> > On Mon, Apr 7, 2014 at 6:30 PM, Jeff Squyres (jsquyres) <
> jsquy...@cisco.com> wrote:
> > Mike --
> >
> > Why did you comment these out?  By definition, --debug output should be
> a LOT of output.
> >
> >
> > On Apr 5, 2014, at 7:27 PM,  wrote:
> >
> > > Author: miked (Mike Dubman)
> > > Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> > > New Revision: 1637
> > > URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> > >
> > > Log:
> > > silence print
> > >
> > > Text files modified:
> > >   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++--
> > >   1 files changed, 2 insertions(+), 2 deletions(-)
> > >
> > > Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> > >
> ==
> > > --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17
> 14:14:47 2014(r1636)
> > > +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05
> 19:27:28 EDT (Sat, 05 Apr 2014)  (r1637)
> > > @@ -331,7 +331,7 @@
> > >
> > > # Check the environment for OMPI_MCA_* values
> > > foreach my $e (keys(%ENV)) {
> > > -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > > +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > > if ($e =~ m/^OMPI_MCA_(\S+)/) {
> > > my $v = $ENV{"OMPI_MCA_$1"};
> > > push(@params, "--env-mca $1 $v");
> > > @@ -339,7 +339,7 @@
> > > }
> > >
> > > $str = join(' ', @params);
> > > -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > > +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > > $str;
> > > }
> > >
> > > ___
> > > mtt-svn mailing list
> > > mtt-...@open-mpi.org
> > > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> >
> >
> > --
> > Jeff Squyres
> > jsquy...@cisco.com
> > For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> >
> > ___
> > mtt-devel mailing list
> > mtt-de...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> >
> > ___
> > mtt-devel mailing list
> > mtt-de...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

Yes.

The intent is that --debug is *very* verbose, and is generally only useful 
when something goes wrong.

I run Cisco's automated MTT with only --verbose.



On Apr 7, 2014, at 6:35 PM, Mike Dubman  wrote:

> ohh.. it is just flooding the log with same data for every test launch.
> 
> maybe we should have verbose level in mtt?
> 
> 
> On Mon, Apr 7, 2014 at 6:30 PM, Jeff Squyres (jsquyres)  
> wrote:
> Mike --
> 
> Why did you comment these out?  By definition, --debug output should be a LOT 
> of output.
> 
> 
> On Apr 5, 2014, at 7:27 PM,  wrote:
> 
> > Author: miked (Mike Dubman)
> > Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> > New Revision: 1637
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> >
> > Log:
> > silence print
> >
> > Text files modified:
> >   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++--
> >   1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> > ==
> > --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17 14:14:47 
> > 2014(r1636)
> > +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05 19:27:28 
> > EDT (Sat, 05 Apr 2014)  (r1637)
> > @@ -331,7 +331,7 @@
> >
> > # Check the environment for OMPI_MCA_* values
> > foreach my $e (keys(%ENV)) {
> > -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > if ($e =~ m/^OMPI_MCA_(\S+)/) {
> > my $v = $ENV{"OMPI_MCA_$1"};
> > push(@params, "--env-mca $1 $v");
> > @@ -339,7 +339,7 @@
> > }
> >
> > $str = join(' ', @params);
> > -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > $str;
> > }
> >
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Mike Dubman]

ohh.. it is just flooding the log with same data for every test launch.

maybe we should have verbose level in mtt?


On Mon, Apr 7, 2014 at 6:30 PM, Jeff Squyres (jsquyres)
wrote:

> Mike --
>
> Why did you comment these out?  By definition, --debug output should be a
> LOT of output.
>
>
> On Apr 5, 2014, at 7:27 PM,  wrote:
>
> > Author: miked (Mike Dubman)
> > Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> > New Revision: 1637
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> >
> > Log:
> > silence print
> >
> > Text files modified:
> >   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++--
> >   1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> >
> ==
> > --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17
> 14:14:47 2014(r1636)
> > +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05
> 19:27:28 EDT (Sat, 05 Apr 2014)  (r1637)
> > @@ -331,7 +331,7 @@
> >
> > # Check the environment for OMPI_MCA_* values
> > foreach my $e (keys(%ENV)) {
> > -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > if ($e =~ m/^OMPI_MCA_(\S+)/) {
> > my $v = $ENV{"OMPI_MCA_$1"};
> > push(@params, "--env-mca $1 $v");
> > @@ -339,7 +339,7 @@
> > }
> >
> > $str = join(' ', @params);
> > -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > $str;
> > }
> >
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

Mike --

Why did you comment these out?  By definition, --debug output should be a LOT 
of output.


On Apr 5, 2014, at 7:27 PM,  wrote:

> Author: miked (Mike Dubman)
> Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> New Revision: 1637
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> 
> Log:
> silence print
> 
> Text files modified: 
>   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++-- 
>
>   1 files changed, 2 insertions(+), 2 deletions(-)
> 
> Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> ==
> --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17 14:14:47 
> 2014(r1636)
> +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05 19:27:28 EDT 
> (Sat, 05 Apr 2014)  (r1637)
> @@ -331,7 +331,7 @@
> 
> # Check the environment for OMPI_MCA_* values
> foreach my $e (keys(%ENV)) {
> -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> if ($e =~ m/^OMPI_MCA_(\S+)/) {
> my $v = $ENV{"OMPI_MCA_$1"};
> push(@params, "--env-mca $1 $v");
> @@ -339,7 +339,7 @@
> }
> 
> $str = join(' ', @params);
> -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> $str;
> }
> 
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] mtt-relay patch - create pid file when run as daemon

[Mike Dubman]

/var/run is only writable to root, but script uses it explicitly.
maybe it is worse to add fallback if non-root user starts mtt-relay.


On Mon, Sep 30, 2013 at 2:08 PM, Christoph Niethammer wrote:

> Hello,
>
> As on many systems init scripts and the handling of services is based on
> pid files I extended the mtt-relay script as follows:
>
> If run with the --daemon option
> * Create file /var/run/mtt-relay.pid  if it does not exist and write the
> pid of the background process into it.
> * exit with return value 1 if /var/run/mtt-relay.pid file exists.
>
> Patch is attached.
>
> Best regards
> Christoph Niethammer
>
> --
>
> Christoph Niethammer
> High Performance Computing Center Stuttgart (HLRS)
> Nobelstrasse 19
> 70569 Stuttgart
>
> Tel: ++49(0)711-685-87203
> email: nietham...@hlrs.de
> http://www.hlrs.de/people/niethammer
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] MTT feature request

[Thomas Naughton]

Hi Jeff,

Cool, no worries.  I just didn't want to forget.

Thanks,
--tjn

 _
  Thomas Naughton  naught...@ornl.gov
  Research Associate   (865) 576-4184


On Thu, 11 Apr 2013, Jeff Squyres (jsquyres) wrote:


Finally done -- sorry for the delay!

On Apr 10, 2013, at 11:18 AM, Thomas Naughton  wrote:


Hi Jeff,

Do you know if the 'email_detailed_report_onfail' request for MTT ever got
a green light?  I just checked and didn't see it in trunk.

Quick refresher summary:
+ Adds new 'email_detailed_report_onfail' flag to TextReporter
  that only sends detailed report if the test was not "success".
  (Similar to existing 'email_detailed_report' flag.)

Thanks,
--tjn

_
 Thomas Naughton  naught...@ornl.gov
 Research Associate   (865) 576-4184


On Thu, 28 Mar 2013, Thomas Naughton wrote:


Hi Jeff,


Sorry for the delay in replying to this; somehow this mail slipped by me.


No problem-o.   :-)

Yes, I just updated local checkout and appears to be just fine
with latest trunk.

Thanks,
--tjn

_
 Thomas Naughton  naught...@ornl.gov
 Research Associate   (865) 576-4184


On Thu, 28 Mar 2013, Jeff Squyres (jsquyres) wrote:


Sorry for the delay in replying to this; somehow this mail slipped by me.

I'm all for what you described. Does your patch still apply to the current 
trunk?  (Texteeporter was just recently split up a bit)

Sent from my phone. No type good.

On Feb 28, 2013, at 10:07 PM, "Thomas Naughton"  wrote:


Hey,

First, thank you for MTT!

I was curious if you would be willing to add a basic feature related to
detailed emails.  Basically, we only want the detailed test results if
something fails.

Initially I thought I could use an () funclet on the
'email_detailed_report' field, but it turns out the summary value used for
"$overall_mtt_status" generated inside the TextFile reporter.

So my solution is to just add another field for this case:

   # If true (1), send the detailed report if there were failures
  email_detailed_report_onfail

Attached is a small patch against mtt-trunk (r1590).

Thanks,
--tjn

_
Thomas Naughton  naught...@ornl.gov
Research Associate   (865) 576-4184








--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1575 - trunk/lib/MTT/Reporter

[Jeff Squyres (jsquyres)]

Wow!  Icky.  That should be reported upstream.


On Jan 15, 2013, at 1:28 PM, Mike Dubman 
 wrote:

> there is a die"" in the MongoDB.connect :(
> 
> On Mon, Jan 14, 2013 at 4:47 PM, Jeff Squyres (jsquyres)  
> wrote:
> That's weird.  Why would an "eval" fix this situation?
> 
> 
> On Jan 14, 2013, at 8:15 AM, svn-commit-mai...@open-mpi.org wrote:
> 
> > Author: miked (Mike Dubman)
> > Date: 2013-01-14 08:15:54 EST (Mon, 14 Jan 2013)
> > New Revision: 1575
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1575
> >
> > Log:
> > fixes #1696
> >
> > Text files modified:
> >   trunk/lib/MTT/Reporter/MTTMongodb.pm | 2 +-
> >   1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/lib/MTT/Reporter/MTTMongodb.pm
> > ==
> > --- trunk/lib/MTT/Reporter/MTTMongodb.pm  Fri Jan  4 14:39:57 2013  
> >   (r1574)
> > +++ trunk/lib/MTT/Reporter/MTTMongodb.pm  2013-01-14 08:15:54 EST (Mon, 
> > 14 Jan 2013)  (r1575)
> > @@ -117,7 +117,7 @@
> >
> >   if($enable_mongo == 1)
> >   {
> > - eval "$conn = MongoDB::Connection->new(host => $url);";
> > + $conn = MongoDB::Connection->new(host => $url);
> >   if(defined($conn))
> >   {
> >   $db = $conn->mlnx_mtt;
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1481 - in trunk: client lib/MTT/Reporter

[Mike Dubman]

Hi,

We are switching from datastore (feature we added a couple of years ago) to
MongoDB NoSQL DB to keep mtt results.

We are adding some "regression" capability based on  MTT and MongoDB
reporter:

- run mtt
- when mtt finishes, extract results for previous runs of the same test
with same parameters
- compare performance metrics and generate regression report (excel)
- attach regression report to the mtt email report

So, we are adding all lego-like utils to support this:

- save results to OO storage (for comfort using from Perl)
- create Analyzers for various well-known tests
- query results, group them and generate regression statistics, place
report into excel (mongo-query.pl)
- Generate report which can be attached to mtt report (breport.pl)


So, we have reporter and query tool for Mongo, which is simple and
customizable.

regards
M


On Wed, Aug 1, 2012 at 2:00 PM, Jeff Squyres  wrote:

> Mike --
>
> MongoDB is a NoSQL thingy, right?
>
> Can you describe this plugin a bit?  Do you guys have some kind of
> reporter for MongoDB?
>
>
> On Aug 1, 2012, at 5:46 AM,  wrote:
>
> > Author: miked (Mike Dubman)
> > Date: 2012-08-01 05:46:03 EDT (Wed, 01 Aug 2012)
> > New Revision: 1481
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1481
> >
> > Log:
> > add modified version mongobquery and MTTMongodb
> >
> > Added:
> >   trunk/client/mongobquery.pl   (contents, props changed)
> >   trunk/lib/MTT/Reporter/MTTMongodb.pm
> >
> > Added: trunk/client/mongobquery.pl
> >
> ==
> > --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> > +++ trunk/client/mongobquery.pl   2012-08-01 05:46:03 EDT (Wed, 01
> Aug 2012)  (r1481)
> > @@ -0,0 +1,1018 @@
> > +#!/usr/bin/perl
> > +#
> > +# Copyright (c) 2009
> > +# $COPYRIGHT$
> > +#
> > +# Additional copyrights may follow
> > +#
> > +# $HEADER$
> > +#
> > +# Now that @INC is setup, bring in the modules
> > +
> > +#use strict;
> > +#use warnings;
> > +use LWP::UserAgent;
> > +use HTTP::Request::Common;
> > +use Data::Dumper;
> > +use File::Basename;
> > +use File::Temp;
> > +use Config::IniFiles;
> > +use YAML::XS;
> > +use MongoDB;
> > +use MongoDB::OID;
> > +use YAML;
> > +use YAML::Syck;
> > +use DateTime;
> > +
> > +###
> > +# Set variables
> > +###
> > +my $module_name=$0;
> > +my $module_path=$0;
> > +
> > +$module_name=~s/([^\/\\]+)$//;
> > +$module_name=$1;
> > +
> > +$module_path=~s/([^\/\\]+)$//;
> > +
> > +
> > +###
> > +# Main block
> > +###
> > +use Getopt::Long qw(:config no_ignore_case);
> > +
> > +my $opt_help;
> > +my $opt_server;
> > +my $opt_username;
> > +my $opt_password;
> > +
> > +my $opt_ping;
> > +my $opt_upload;
> > +my $opt_query;
> > +my $opt_view;
> > +my $opt_admin;
> > +
> > +my @opt_data;
> > +my @opt_raw;
> > +
> > +my $opt_gqls;
> > +my @opt_gqlf;
> > +my @opt_section;
> > +my $opt_dir;
> > +my $opt_no_raw;
> > +
> > +my $opt_dstore;
> > +my $opt_info;
> > +my $opt_format;
> > +my $opt_mailto;
> > +my $opt_regression_from;
> > +my $opt_regression_to;
> > +my $opt_regression_step;
> > +
> > +my @opt_newuser;
> > +
> > +GetOptions ("help|h" => \$opt_help,
> > +"server|a=s" => \$opt_server,
> > +"username|u=s" => \$opt_username,
> > +"password|p=s" => \$opt_password,
> > +"ping" => \$opt_ping,
> > +"upload" => \$opt_upload,
> > +"query" => \$opt_query,
> > +"view" => \$opt_view,
> > +"admin" => \$opt_admin,
> > +
> > +"data|S=s" => \@opt_data,
> > +"raw|R=s" => \@opt_raw,
> > +
> > +"gqls|L=s" => \$opt_gqls,
> > +"gqlf|F=s" => \@opt_gqlf,
> > +"section|T=s" => \@opt_section,
> > +"dir|O=s" => \$opt_dir,
> > +"no-raw" => \$opt_no_raw,
> > +
> > +"dstore|D" => \$opt_dstore,
> > +"info|I=s" => \$opt_info,
> > +"format|V=s" => \$opt_format,
> > +"email|e=s" => \$opt_mailto,
> > +
> > +"newuser=s{3,5}" => \@opt_newuser,
> > +
> > + "regression-from=s" => \$opt_regression_from,
> > + "regression-to=s" => \$opt_regression_to,
> > + "regression-step=s" => \$opt_regression_step
> > +);
> > +
> > +
> > +my $url = ();
> > +my $username = ();
> > +my $password = ();
> > +
> > +$url = $opt_server ? $opt_server : "http://bgate.mellanox.com:27017;;
> > +$url =~ s/http:\/\///;
> > +$username = $opt_username ? $opt_username : "admin";
> > +$password = $opt_password ? $opt_password : "";
> > +
> > +my %conf = ('url' => "$url\/client",
> > +'username' => $username,
> > +  

Re: [MTT devel] [MTT svn] svn:mtt-svn r1481 - in trunk: client lib/MTT/Reporter

[Jeff Squyres]

Mike --

MongoDB is a NoSQL thingy, right?

Can you describe this plugin a bit?  Do you guys have some kind of reporter for 
MongoDB?


On Aug 1, 2012, at 5:46 AM,  wrote:

> Author: miked (Mike Dubman)
> Date: 2012-08-01 05:46:03 EDT (Wed, 01 Aug 2012)
> New Revision: 1481
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1481
> 
> Log:
> add modified version mongobquery and MTTMongodb
> 
> Added:
>   trunk/client/mongobquery.pl   (contents, props changed)
>   trunk/lib/MTT/Reporter/MTTMongodb.pm
> 
> Added: trunk/client/mongobquery.pl
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ trunk/client/mongobquery.pl   2012-08-01 05:46:03 EDT (Wed, 01 Aug 
> 2012)  (r1481)
> @@ -0,0 +1,1018 @@
> +#!/usr/bin/perl
> +#
> +# Copyright (c) 2009
> +# $COPYRIGHT$
> +# 
> +# Additional copyrights may follow
> +# 
> +# $HEADER$
> +#
> +# Now that @INC is setup, bring in the modules
> +
> +#use strict;
> +#use warnings;
> +use LWP::UserAgent;
> +use HTTP::Request::Common;
> +use Data::Dumper;
> +use File::Basename;
> +use File::Temp;
> +use Config::IniFiles;
> +use YAML::XS;
> +use MongoDB;
> +use MongoDB::OID;
> +use YAML;
> +use YAML::Syck;
> +use DateTime;
> +
> +###
> +# Set variables
> +###
> +my $module_name=$0;
> +my $module_path=$0;
> +
> +$module_name=~s/([^\/\\]+)$//;
> +$module_name=$1;
> + 
> +$module_path=~s/([^\/\\]+)$//;
> +
> +
> +###
> +# Main block
> +###
> +use Getopt::Long qw(:config no_ignore_case);
> +
> +my $opt_help;
> +my $opt_server;
> +my $opt_username;
> +my $opt_password;
> +
> +my $opt_ping;
> +my $opt_upload;
> +my $opt_query;
> +my $opt_view;
> +my $opt_admin;
> +
> +my @opt_data;
> +my @opt_raw;
> +
> +my $opt_gqls;
> +my @opt_gqlf;
> +my @opt_section;
> +my $opt_dir;
> +my $opt_no_raw;
> +
> +my $opt_dstore;
> +my $opt_info;
> +my $opt_format;
> +my $opt_mailto;
> +my $opt_regression_from;
> +my $opt_regression_to;
> +my $opt_regression_step;
> +
> +my @opt_newuser;
> +
> +GetOptions ("help|h" => \$opt_help,
> +"server|a=s" => \$opt_server,
> +"username|u=s" => \$opt_username,
> +"password|p=s" => \$opt_password,
> +"ping" => \$opt_ping,
> +"upload" => \$opt_upload,
> +"query" => \$opt_query,
> +"view" => \$opt_view,
> +"admin" => \$opt_admin,
> +
> +"data|S=s" => \@opt_data,
> +"raw|R=s" => \@opt_raw, 
> +
> +"gqls|L=s" => \$opt_gqls,
> +"gqlf|F=s" => \@opt_gqlf,
> +"section|T=s" => \@opt_section,
> +"dir|O=s" => \$opt_dir,
> +"no-raw" => \$opt_no_raw,
> +
> +"dstore|D" => \$opt_dstore,
> +"info|I=s" => \$opt_info,
> +"format|V=s" => \$opt_format,
> +"email|e=s" => \$opt_mailto,
> +
> +"newuser=s{3,5}" => \@opt_newuser,
> +
> + "regression-from=s" => \$opt_regression_from,
> + "regression-to=s" => \$opt_regression_to,
> + "regression-step=s" => \$opt_regression_step
> +);
> +
> +
> +my $url = ();
> +my $username = ();
> +my $password = ();
> +
> +$url = $opt_server ? $opt_server : "http://bgate.mellanox.com:27017;;
> +$url =~ s/http:\/\///;
> +$username = $opt_username ? $opt_username : "admin";
> +$password = $opt_password ? $opt_password : "";
> +
> +my %conf = ('url' => "$url\/client",
> +'username' => $username,
> +'password' => $password
> +);
> +
> +if ($opt_help)
> +{
> +my $action = '';
> + 
> +$action = 'ping' if ($opt_ping);
> +$action = 'upload' if ($opt_upload);
> +$action = 'query' if ($opt_query);
> +$action = 'view' if ($opt_view);
> +$action = 'admin' if ($opt_admin);
> +
> +help($action);
> +
> +exit;
> +}
> +elsif ($opt_ping)
> +{
> + #ping( \%conf ); 
> + #print $url," url\n";
> + my $conn = MongoDB::Connection->new(host => $url );
> + if($conn != 0)
> + {
> + print"\n\nping: success\n\n";
> + }
> +}
> +elsif ($opt_upload)
> +{
> +if ($#opt_data < 0) 
> +{
> +help('upload');
> +}
> + my @data = split(/,/,join(',',@opt_data)) if (@opt_data);
> + my @raw = split(/,/,join(',',@opt_raw)) if (@opt_raw);
> +
> +# Check if files existed
> + verify_opt_file( @data );
> + verify_opt_file( @raw );
> +
> + $conf{data} = \@data;
> + $conf{raw} = \@raw;  
> +
> + upload( \%conf ); 
> +}
> +elsif ($opt_query)
> +{
> +my $gql = ();
> +if ($opt_gqls) 
> +{
> +$gql = $opt_gqls;
> +}
> +

Re: [MTT devel] [MTT svn] svn:mtt-svn r1433

[Jeff Squyres]

Yummy -- thanks Mike!

Can you add this field (and any others you have added) to the wiki?

https://svn.open-mpi.org/trac/mtt/wiki/MTTINIFileFields

This is our *only* documentation; it's important to keep it up to date.

Thanks!


On Jan 25, 2012, at 6:02 AM, mi...@osl.iu.edu wrote:

> Author: miked
> Date: 2012-01-25 06:02:47 EST (Wed, 25 Jan 2012)
> New Revision: 1433
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1433
> 
> Log:
> * Introduce mtt.break_threshold_timeout_and_fail ini param to specify % of 
> overall failed tests to trigger stop execution.
> 
> 
> Text files modified: 
>   trunk/lib/MTT/Reporter/TextFile.pm | 9 -
>
>   trunk/lib/MTT/Test/Run.pm  |30  
>
>   trunk/lib/MTT/Test/RunEngine.pm|60 
> +++ 
>   trunk/lib/MTT/Util.pm  | 6  
>
>   trunk/lib/MTT/Values.pm| 3 +
>
>   5 files changed, 86 insertions(+), 22 deletions(-)
> 
> Modified: trunk/lib/MTT/Reporter/TextFile.pm
> ==
> --- trunk/lib/MTT/Reporter/TextFile.pm(original)
> +++ trunk/lib/MTT/Reporter/TextFile.pm2012-01-25 06:02:47 EST (Wed, 
> 25 Jan 2012)
> @@ -230,8 +230,13 @@
> my $filename = "All_phase-summary.txt";
> my $file = "$dirname/" . MTT::Files::make_safe_filename("$filename");
> 
> -my $body = join("\n", ($summary_header, $table->render, $perf_stat, 
> $summary_footer));
> -
> +my $body;
> +if ($MTT::Globals::Internals->{is_stopped_on_break_threshold}){
> +$body = join("\n", ($summary_header, $table->render, $perf_stat, 
> $MTT::Globals::Internals->{stopped_on_break_threshold_message}, 
> $summary_footer));
> +}
> +else{
> +$body = join("\n", ($summary_header, $table->render, $perf_stat, 
> $summary_footer));
> +}   
> print $body;
> _output_results($file, $body);
> 
> 
> Modified: trunk/lib/MTT/Test/Run.pm
> ==
> --- trunk/lib/MTT/Test/Run.pm (original)
> +++ trunk/lib/MTT/Test/Run.pm 2012-01-25 06:02:47 EST (Wed, 25 Jan 2012)
> @@ -64,8 +64,11 @@
> #--
> 
> sub Run {
> -my ($ini, $ini_full, $install_dir, $runs_data_dir, $force) = @_;
> +my ($ini, $ini_full, $install_dir, $runs_data_dir, $force, 
> $count_total_tests_number) = @_;
> 
> +if ($count_total_tests_number ne "yes"){
> +Run($ini, $ini_full, $install_dir, $runs_data_dir, $force,"yes");
> +}
> # Save the environment
> my %ENV_SAVE = %ENV;
> 
> @@ -188,7 +191,7 @@
> _do_run($ini, $section, $test_build, 
> $mpi_get, $mpi_install,
> $install_dir, $runs_data_dir, 
> -$force);
> +
> $force,$count_total_tests_number);
> delete 
> $MTT::Globals::Internals->{mpi_get_name};
> delete 
> $MTT::Globals::Internals->{mpi_install_name};
> delete 
> $MTT::Globals::Internals->{test_get_name};
> @@ -196,12 +199,24 @@
> delete 
> $MTT::Globals::Internals->{test_run_name};
> %ENV = %ENV_SAVE;
> }
> +last
> +if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> +last
> +if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> +last
> +if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> +last
> +if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> +last
> +if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> }
> +last
> +if ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> 
> Verbose("*** Run test phase complete\n");
> @@ -211,7 +226,7 @@
> 
> sub _do_run {
> my ($ini, $section, $test_build, $mpi_get, $mpi_install, $install_dir, 
> -$runs_data_dir, $force) = @_;
> +$runs_data_dir, $force, $count_total_tests_number) = @_;
> 
> # Simple section name
> my $simple_section = GetSimpleSection($section);
> @@ -528,9 +543,13 @@
> # If we got a list of tests to run, 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1377

[Jeff Squyres]

Ok, that's what I thought, but your comment doesn't seem to match this:

# mtt_switch(@np@, 9, "return1", 100, return2", "default", 0);

You have an extra "return" in there, and the 5th argument doesn't seem to be 
quoted properly.



On Jan 3, 2011, at 8:21 AM, Mike Dubman wrote:

> Hi,
> it is c-style *switch* replacement, to simplify statements like this:
> 
> transport   =  (('@cycle@', '1'), '@btl_openib@' ,\
>   (('@cycle@', '2'), '@btl_openib@',\
> (('@cycle@', '3'), '@btl_eth10g@',\
>   (('@cycle@', '4'), '@btl_eth10g@',\
> (('@cycle@', '3a'), '@btl_eth10g@ -x 
> custom_noswitchconnect=1',\
>   (('@cycle@', '4a'), '@btl_eth10g@ -x 
> custom_noswitchconnect=1',\
> (('@cycle@', '5'), '@btl_eth10g@',\
>   (('@cycle@', '6'), '@btl_eth10g@',\
> (('@cycle@', '7'), '@btl_eth10g@',\
>   (('@cycle@', '8'), 
> '@btl_eth10g@',\
> (('@cycle@', '11'), 
> '@btl_eth10g@',\
>   (('@cycle@', '12'), 
> '@btl_eth10g@',\
>'@btl_rdmaoe@'\
>  )\
>)\
>  )\
>)\
>  )\
>)\
>  )\
>)\
>  )\
>)\
> )\
>   )
> 
> which can be rewritten as:
> 
> transport = mtt_switch(@cycle@, 1, @btl_openib@, 2, @btl_eth10g@, ...)
> 
> Will update wiki as well.
> 
> 
> Regards
> 
> 
> 
> On Mon, Jan 3, 2011 at 2:54 PM, Jeff Squyres  wrote:
> Mike --
> 
> Can you explain this one a little?  I don't understand the example you gave 
> in the comment.
> 
> Also, are you adding all the new funclets to the wiki documentation?
> 
> 
> 
> On Dec 29, 2010, at 3:52 AM, mi...@osl.iu.edu wrote:
> 
> > Author: miked
> > Date: 2010-12-29 03:52:24 EST (Wed, 29 Dec 2010)
> > New Revision: 1377
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1377
> >
> > Log:
> > added mtt_switch to simplify if-then-else cases in ini files
> >
> > Text files modified:
> >   trunk/lib/MTT/Values/Functions.pm |21 +
> >   1 files changed, 21 insertions(+), 0 deletions(-)
> >
> > Modified: trunk/lib/MTT/Values/Functions.pm
> > ==
> > --- trunk/lib/MTT/Values/Functions.pm (original)
> > +++ trunk/lib/MTT/Values/Functions.pm 2010-12-29 03:52:24 EST (Wed, 29 Dec 
> > 2010)
> > @@ -3163,4 +3163,25 @@
> >   return $x->{result_stdout};
> > }
> >
> > +#
> > +# Poor man switch statement
> > +# Example: mtt_switch(@np@, 9, "return1", 100, return2", "default", 0);
> > +#
> > +
> > +sub mtt_switch
> > +{
> > +my ($var, %cases) = @_;
> > +
> > +if ($cases{$var}) {
> > +return $cases{$var};
> > +}
> > +
> > +if ($cases{'default'}) {
> > +return $cases{'default'};
> > +}
> > +
> > +Debug("ERROR: Not found case for $var\n");
> > +}
> > +
> > +
> > 1;
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTT GDS -- one more...

[Jeff Squyres]

On Feb 21, 2010, at 5:52 AM, Mike Dubman wrote:

> > > 2. In reading through the Google Appengine docs, the GDS stuff looks like
> > we mainly can access the data through GQL.  I don't see any mention of doing
> > map/reduce kinds of computations (Ethan and I were talking on the phone
> > today about MTT Appengine possibilities).  I'm new to all this stuff, so
> > it's quite possible that a) I missed it, or b) I just don't understand what
> > I'm seeing/reading yet.  Or does GQL do map/reduce on the back end to do its
> > magic?  Is GQL the main/only way we have to access GDS?
> >
> > As far as I and Igor know there are no way of doing Map/Reduce with GDS. And
> > GQL (or filters which is practically synonym) is the main and only way to
> > access GDS data.
> 
> btw, afaik - the gds is using mapreduce explicitly on the backend - one needs 
> do nothing in order to ENABLE it: just use their API and the framework will 
> do the rest.

Hm; ok.  So they only give us limited access to map reduce -- via GQL?  (i.e., 
we can only do what GQL allows us to do)

For example, I'd love to be able to specify my own reductions in order to do 
some real data mining, parsing through the data, etc.  Specifically, I'd like 
to be able to exploit the whole "bring the computation to the data" philosophy 
of map reduce, because MTT data can be *huge*.  But I don't see how to do this 
with GQL...?

Am I missing something?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTT GDS -- one more...

[Andrew Senin]

Hello Jeff, 

I worked with Igor on the GDS framework (although Igor knows more tech
details than me). Let me put my two cents to the discussion.

> 1. It looks like the main benefits of using the Google App Engine --
specifically for MTT -- is that we can use the GDS and/or we can host an
application on their web servers.  Is that correct?

I think yes. Also GDS should work faster than a relational DB on large
amounts of data.

> 2. In reading through the Google Appengine docs, the GDS stuff looks like
we mainly can access the data through GQL.  I don't see any mention of doing
map/reduce kinds of computations (Ethan and I were talking on the phone
today about MTT Appengine possibilities).  I'm new to all this stuff, so
it's quite possible that a) I missed it, or b) I just don't understand what
I'm seeing/reading yet.  Or does GQL do map/reduce on the back end to do its
magic?  Is GQL the main/only way we have to access GDS?

As far as I and Igor know there are no way of doing Map/Reduce with GDS. And
GQL (or filters which is practically synonym) is the main and only way to
access GDS data.

> 3. Is there a reason that MTTGDS.pm doesn't use the python API to directly
talk to GDS?  I.e., what is the rationale for using a web app on appengine?
Is the web app doing stuff that we can't do at the client?  Ditto for
bquery.pl and breport.pl.  (these questions are partially fueled by my
curiosity and concern about why we're using so much CPU at Google)

There are a few reasons of doing it. The first is speed. When we post new
data we firstly try to find if there is a copy of corresponding MpiInfo,
ClustreInfo and other *Info classes. If we did it directly from client
scripts the delays would be higher (depending on Internet connection speed).
Price of it is additional CPU cycles on google servers. The second and more
important is that when we have such logic on server we (instead of GDS
clients) are responsible for maintaining correct structure of links between
objects. If such logic was implemented on client side user could (by mistake
or on purpose) break links between objects.

Regards, 
Andrew Senin

-Original Message-
List-Post: mtt-devel@lists.open-mpi.org
Date: Thu, 11 Feb 2010 21:43:21 -0500
From: Jeff Squyres 
Subject: [MTT devel] MTT GDS -- one more...
To: Development list for the MPI Testing Tool 
Message-ID: <8a556b10-1618-47ea-96a9-33f22aecd...@cisco.com>
Content-Type: text/plain; charset=us-ascii

Heh... even more questions...

(BTW, Ethan and I have asked s many questions that if it helps, I can
setup a webex and we can all discuss this in person rather than via
1,000,000 annoying emails from us.  :-)  Webex can call you; no one will
need to pay for an international call)

1. It looks like the main benefits of using the Google App Engine --
specifically for MTT -- is that we can use the GDS and/or we can host an
application on their web servers.  Is that correct?

2. In reading through the Google Appengine docs, the GDS stuff looks like we
mainly can access the data through GQL.  I don't see any mention of doing
map/reduce kinds of computations (Ethan and I were talking on the phone
today about MTT Appengine possibilities).  I'm new to all this stuff, so
it's quite possible that a) I missed it, or b) I just don't understand what
I'm seeing/reading yet.  Or does GQL do map/reduce on the back end to do its
magic?  Is GQL the main/only way we have to access GDS?

3. Is there a reason that MTTGDS.pm doesn't use the python API to directly
talk to GDS?  I.e., what is the rationale for using a web app on appengine?
Is the web app doing stuff that we can't do at the client?  Ditto for
bquery.pl and breport.pl.  (these questions are partially fueled by my
curiosity and concern about why we're using so much CPU at Google)

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] mtt not working on sles 11up2 perl 5.10.0

[Jeff Squyres]

Hmm.  This is trunk, I assume?  If so, Values.pm:107 is:


my $ret = _eval_func($func_name, $func_args);

# If we got a string back, append the remaining and loop
# around looking for more 
--> if (ref($ret) eq "") {
-

So it's the return of evaluating the function.

Your output says ">>> Using group_reports" , but I don't see a "group_reports" 
in mini.ini...?

I think what I would do here is try to figure out what function it was trying 
to invoke -- perhaps something returned undef instead of a value...?  If so, 
perhaps we should protect that if statement if a check to see if $ret is 
defined...?


On Jan 27, 2010, at 4:36 AM, Mike Dubman wrote:

> 
> 
> Hello guys,
> 
> 
> mtt fails on sles11up2 with perl version 5.10.0 but works on other distros as 
> a charm.
> 
> The same minimalistic ini file which works on other distro`s fails on sles 
> with error:
> 
> >> Test Run [osu]
> >> Running with [open mpi] / [1.3.3] / [openmpi]
>Using MPI Details [open mpi] with MPI Install [openmpi]
> >>> Using group_reports
> Can't use string ("2") as an ARRAY ref while "strict refs" in use at 
> /hpc/home/USERS/mttuserqa/work/svn/ompi/mtt/trunk/lib/MTT/Values.pm line 107.
> 
> Do you have any ideas what it may be?
> 
> P.S. mini.ini is attached.
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com

Re: [MTT devel] [MTT svn] svn:mtt-svn r1320

[Mike Dubman]

it seems it can be retired. executable() covers more cases.
shell() can be the alias of executable() for backwards compatibility.

Also, DoCommand::CmdScript should be changed to DoCommand::Cmd inside
executable() to really cover more cases.
regards

Mike

On Tue, Sep 29, 2009 at 8:35 PM, Ethan Mallove wrote:

> Should () be deprecated? It looks awfully similar to
> ().
>
> -Ethan
>
> On Tue, Sep/29/2009 08:34:44AM, mi...@osl.iu.edu wrote:
> > Author: miked
> > Date: 2009-09-29 08:34:44 EDT (Tue, 29 Sep 2009)
> > New Revision: 1320
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1320
> >
> > Log:
> > applied Jeff,Ethan comments:
> > 1. rename on_stop,on_start to after_mtt_start_exec, before_mtt_start_exec
> > 2. treat *_mtt_start_exec params in the same way like others
> before/after_* params
> > 3. rename shell_script to executable
> > 4. fix DoCommand:CmdScript() to recognize shebang chars and do not add
> ":\n" if #! is present
> >
> >
> > Text files modified:
> >trunk/client/mtt  |15 +--
> >trunk/lib/MTT/DoCommand.pm| 5 +++--
> >trunk/lib/MTT/Values/Functions.pm | 2 +-
> >3 files changed, 17 insertions(+), 5 deletions(-)
> >
> > Modified: trunk/client/mtt
> >
> ==
> > --- trunk/client/mtt  (original)
> > +++ trunk/client/mtt  2009-09-29 08:34:44 EDT (Tue, 29 Sep 2009)
> > @@ -496,7 +496,8 @@
> >  MTT::Lock::Init($ini);
> >
> >  # execute on_start callback if exists
> > -_process_get_value_option("mtt,on_start", $ini);
> > + _do_step($ini, "mtt", "before_mtt_start_exec");
> > +
> >
> >  # Set the logfile, if specified
> >
> > @@ -565,7 +566,7 @@
> >  }
> >
> >  # execute on_stop callback if exists
> > -_process_get_value_option("mtt,on_stop", $ini);
> > + _do_step($ini, "mtt", "after_mtt_start_exec");
> >
> >  # Shut down locks
> >
> > @@ -737,3 +738,13 @@
> >  print "$value\n";
> >  }
> >  }
> > +
> > +# Run cmd, specified in the non Test* sections
> > +sub _do_step {
> > + my ($ini, $section,$param) = @_;
> > + my $cmd = $ini->val($section, $param);
> > + if ( defined $cmd ) {
> > + my $x = MTT::DoCommand::RunStep(1, $cmd, -1, $ini,
> $section, $param);
> > + Verbose("  Output: $x->{result_stdout}\n")
> > + }
> > +}
> >
> > Modified: trunk/lib/MTT/DoCommand.pm
> >
> ==
> > --- trunk/lib/MTT/DoCommand.pm(original)
> > +++ trunk/lib/MTT/DoCommand.pm2009-09-29 08:34:44 EDT (Tue, 29
> Sep 2009)
> > @@ -794,9 +794,10 @@
> >  # protects against a common funclet syntax error.
> >  # We can safely do this since "foo" (literally, with
> >  # quotes included) would never be a valid shell command.
> > -$cmds =~ s/\"$//
> > -if ($cmds =~ s/^\"//);
> > +$cmds =~ s/\"$// if ($cmds =~ s/^\"//);
> >
> > +
> > + print $fh ":\n" if ($cmds !~ /^\s*\#\!/); # no shell specified -
> use default
> >  print $fh "$cmds\n";
> >  close($fh);
> >  chmod(0700, $filename);
> >
> > Modified: trunk/lib/MTT/Values/Functions.pm
> >
> ==
> > --- trunk/lib/MTT/Values/Functions.pm (original)
> > +++ trunk/lib/MTT/Values/Functions.pm 2009-09-29 08:34:44 EDT (Tue, 29
> Sep 2009)
> > @@ -3038,7 +3038,7 @@
> >  #
> >  #
> >
> > -sub shell_script {
> > +sub executable {
> >   my ($cmd_section, $cmd_param) = @_;
> >   my $cmd = _ini_val($cmd_section, $cmd_param);
> >   my $x = MTT::DoCommand::CmdScript(1, $cmd);
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Jeff Squyres]

On Sep 28, 2009, at 4:20 PM, Ethan Mallove wrote:


I was commenting on both r1304 and r1319.

These INI params:

  on_start
  on_stop

are very similar to these INI params:

  after_each_exec
  before_any_exec
  after_all_exec

My thought was that it would make sense for them to use a similar
naming scheme and implementation (e.g., use suffix "_exec" and be
passed to DoCommand::Cmd()).



Agreed.

--
Jeff Squyres
jsquy...@cisco.com

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Ethan Mallove]

On Fri, Sep/25/2009 04:59:11PM, Jeff Squyres wrote:
> I'm not sure what you mean -- I thought they added a funclect, not a 
> field...?

I was commenting on both r1304 and r1319. 

These INI params:

  on_start
  on_stop

are very similar to these INI params: 

  after_each_exec
  before_any_exec
  after_all_exec

My thought was that it would make sense for them to use a similar
naming scheme and implementation (e.g., use suffix "_exec" and be
passed to DoCommand::Cmd()).

-Ethan

>
> On Sep 24, 2009, at 3:09 PM, Ethan Mallove wrote:
>
>> I think on_stop should conform more to these params:
>>
>>   after_each_exec
>>   before_any_exec
>>   after_all_exec
>>
>> E.g.,
>>
>>   before_mtt_start_exec
>>   after_mtt_start_exec
>>
>> Then have &_process_get_value_option() call DoCommand. Note, DoCommand
>> is aware of hashbangs (see &_contains_shell_script_characters()), so
>> _script() might be redundant.
>>
>> -Ethan
>>
>>
>> On Thu, Sep/24/2009 07:46:40PM, Mike Dubman wrote:
>> >Hey Jeff,
>> >
>> >On Thu, Sep 24, 2009 at 4:02 PM, Jeff Squyres  
>> wrote:
>> >
>> >  The DoCommand.pm sub added ":\n" to the beginning to force the 
>> Bourne
>> >  shell interpreter. *This is necessary for some cases where an
>> >  interpreter is not otherwise specified.
>> >
>> >Im not familiar with :\n semantics, how does it force Bourne shell 
>> and
>> >what it actually does :)? (seems like leftovers from 1960)
>> >I think when interpreter is not explicitly specified - the default 
>> user`s
>> >shell is used.
>> >see in the DoCommand::Cmd() . it check the buffer`s* first line 
>> for
>> >#!/... semantic and if found - saves buffer to file, adds +x perm,* 
>> and
>> >just executes it as a regular script.
>> >
>> >When I passed a buffer with shell commands but 1st line was not 
>> #!/bin/sh
>> >- it* failed with syntax errors.
>> >
>> >*
>> >
>> >  I see why you did it -- you want the ability to add your own 
>> interpreter
>> >  in _script(). *Why not either make a parameter to add the 
>> ":\n" or
>> >  not, or better yet, have DoCommand.pm analyze the beginning of the
>> >  string and if it contains "^:\n" or "^#!", then don't add anything. 
>> *But
>> >  if it doesn't contain either of those, then prefix it with ":\n".*
>> >
>> >  How does that sound?
>> >
>> >sounds good!
>> >
>> >  Also, is "_script()" a good name? *If you can specify your 
>> own
>> >  interpreter, it might not be a shell script. *How about 
>> ()?
>> >
>> >ok - () it will be!
>> >*
>> >
>> >regards
>> >
>> >Mike
>> >
>> >  On Sep 24, 2009, at 8:06 AM, mi...@osl.iu.edu wrote:
>> >
>> >Author: miked
>> >Date: 2009-09-24 08:06:04 EDT (Thu, 24 Sep 2009)
>> >New Revision: 1319
>> >URL: https://svn.open-mpi.org/trac/mtt/changeset/1319
>> >
>> >Log:
>> >bug fix: CmdScript() - no need to add extra chars "\n:" to the 
>> start
>> >of shell script file
>> >new funclet: shell_script(section,param)
>> >
>> >Text files modified:
>> >*trunk/lib/MTT/DoCommand.pm * * * *| * * 2 +-
>> >*trunk/lib/MTT/Values/Functions.pm | * *19 +++
>> >*2 files changed, 20 insertions(+), 1 deletions(-)
>> >
>> >Modified: trunk/lib/MTT/DoCommand.pm
>> >
>> ==
>> >--- trunk/lib/MTT/DoCommand.pm *(original)
>> >+++ trunk/lib/MTT/DoCommand.pm *2009-09-24 08:06:04 EDT (Thu, 24 
>> Sep
>> >2009)
>> >@@ -797,7 +797,7 @@
>> >* *$cmds =~ s/\"$//
>> >* * * *if ($cmds =~ s/^\"//);
>> >
>> >- * *print $fh ":\n$cmds\n";
>> >+ * *print $fh "$cmds\n";
>> >* *close($fh);
>> >* *chmod(0700, $filename);
>> >
>> >Modified: trunk/lib/MTT/Values/Functions.pm
>> >
>> ==
>> >--- trunk/lib/MTT/Values/Functions.pm * (original)
>> >+++ trunk/lib/MTT/Values/Functions.pm * 2009-09-24 08:06:04 EDT 
>> (Thu,
>> >24 Sep 2009)
>> >@@ -3026,4 +3026,23 @@
>> >* *return md5_hex($str);
>> >}
>> >
>> >+# Run shell commands as a script, i.e
>> >+#
>> >+# [mtt]
>> >+# myscript=<> >+# #!/bin/sh
>> >+# pwd
>> >+# ls
>> >+# EOT
>> >+# on_stop=_script("mtt",myscript)
>> >+#
>> >+#
>> >+
>> >+sub shell_script {
>> >+ * * * my ($cmd_section, $cmd_param) = @_;
>> >+ * * * my $cmd = _ini_val($cmd_section, $cmd_param);
>> >+ * * * my $x = MTT::DoCommand::CmdScript(1, $cmd);
>> >+ * * * return $x->{result_stdout};
>> >+}
>> >+
>> >1;
>> >___
>> >mtt-svn mailing 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Mike Dubman]

On Fri, Sep 25, 2009 at 10:08 PM, Jeff Squyres  wrote:

> On Sep 24, 2009, at 12:46 PM, Mike Dubman wrote:
>
>  Im not familiar with :\n semantics, how does it force Bourne shell and
>> what it actually does :)? (seems like leftovers from 1960)
>>
>
> Yes, it might be left over from 1960.  :-)  But the nice thing is that you
> then don't have to identify /bin/sh or /usr/bin/sh.  It's convenient and it
> works everywhere.


Found some info re ":\n" as a shebang line:
...
':' was actually the first comment character.
All shells I tried still recognize it as such, so it is not obsolete, but
perhaps slightly deprecated.

The first versions of csh used '#' as a comment and used the presence of one
comment character or the other to decide which shell to run (assuming it was
given a text file with the execute bit set). This was before the advent of
the kernel-based #! "magic number"
The early "/bin/sh" versions assumed they were the only shell on the system
and had no need to choose an interpreter.
...

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Jeff Squyres]

I'm not sure what you mean -- I thought they added a funclect, not a  
field...?


On Sep 24, 2009, at 3:09 PM, Ethan Mallove wrote:


I think on_stop should conform more to these params:

  after_each_exec
  before_any_exec
  after_all_exec

E.g.,

  before_mtt_start_exec
  after_mtt_start_exec

Then have &_process_get_value_option() call DoCommand. Note, DoCommand
is aware of hashbangs (see &_contains_shell_script_characters()), so
_script() might be redundant.

-Ethan


On Thu, Sep/24/2009 07:46:40PM, Mike Dubman wrote:
>Hey Jeff,
>
>On Thu, Sep 24, 2009 at 4:02 PM, Jeff Squyres  
 wrote:

>
>  The DoCommand.pm sub added ":\n" to the beginning to force  
the Bourne

>  shell interpreter. *This is necessary for some cases where an
>  interpreter is not otherwise specified.
>
>Im not familiar with :\n semantics, how does it force Bourne  
shell and

>what it actually does :)? (seems like leftovers from 1960)
>I think when interpreter is not explicitly specified - the  
default user`s

>shell is used.
>see in the DoCommand::Cmd() . it check the buffer`s* first  
line for
>#!/... semantic and if found - saves buffer to file, adds +x  
perm,* and

>just executes it as a regular script.
>
>When I passed a buffer with shell commands but 1st line was not  
#!/bin/sh

>- it* failed with syntax errors.
>
>*
>
>  I see why you did it -- you want the ability to add your own  
interpreter
>  in _script(). *Why not either make a parameter to add  
the ":\n" or
>  not, or better yet, have DoCommand.pm analyze the beginning  
of the
>  string and if it contains "^:\n" or "^#!", then don't add  
anything. *But
>  if it doesn't contain either of those, then prefix it with ": 
\n".*

>
>  How does that sound?
>
>sounds good!
>
>  Also, is "_script()" a good name? *If you can specify  
your own
>  interpreter, it might not be a shell script. *How about  
()?

>
>ok - () it will be!
>*
>
>regards
>
>Mike
>
>  On Sep 24, 2009, at 8:06 AM, mi...@osl.iu.edu wrote:
>
>Author: miked
>Date: 2009-09-24 08:06:04 EDT (Thu, 24 Sep 2009)
>New Revision: 1319
>URL: https://svn.open-mpi.org/trac/mtt/changeset/1319
>
>Log:
>bug fix: CmdScript() - no need to add extra chars "\n:" to  
the start

>of shell script file
>new funclet: shell_script(section,param)
>
>Text files modified:
>*trunk/lib/MTT/DoCommand.pm * * * *| * * 2 +-
>*trunk/lib/MTT/Values/Functions.pm | * *19 + 
++

>*2 files changed, 20 insertions(+), 1 deletions(-)
>
>Modified: trunk/lib/MTT/DoCommand.pm
> 
= 
= 
= 
= 
= 
= 
= 
= 
==

>--- trunk/lib/MTT/DoCommand.pm *(original)
>+++ trunk/lib/MTT/DoCommand.pm *2009-09-24 08:06:04 EDT  
(Thu, 24 Sep

>2009)
>@@ -797,7 +797,7 @@
>* *$cmds =~ s/\"$//
>* * * *if ($cmds =~ s/^\"//);
>
>- * *print $fh ":\n$cmds\n";
>+ * *print $fh "$cmds\n";
>* *close($fh);
>* *chmod(0700, $filename);
>
>Modified: trunk/lib/MTT/Values/Functions.pm
> 
= 
= 
= 
= 
= 
= 
= 
= 
==

>--- trunk/lib/MTT/Values/Functions.pm * (original)
>+++ trunk/lib/MTT/Values/Functions.pm * 2009-09-24 08:06:04  
EDT (Thu,

>24 Sep 2009)
>@@ -3026,4 +3026,23 @@
>* *return md5_hex($str);
>}
>
>+# Run shell commands as a script, i.e
>+#
>+# [mtt]
>+# myscript=<+# #!/bin/sh
>+# pwd
>+# ls
>+# EOT
>+# on_stop=_script("mtt",myscript)
>+#
>+#
>+
>+sub shell_script {
>+ * * * my ($cmd_section, $cmd_param) = @_;
>+ * * * my $cmd = _ini_val($cmd_section, $cmd_param);
>+ * * * my $x = MTT::DoCommand::CmdScript(1, $cmd);
>+ * * * return $x->{result_stdout};
>+}
>+
>1;
>___
>mtt-svn mailing list
>mtt-...@open-mpi.org
>http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>
>  --
>  Jeff Squyres
>  jsquy...@cisco.com
>
>  ___
>  mtt-devel mailing list
>  mtt-de...@open-mpi.org
>  http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> References
>
>Visible links
>. mailto:jsquy...@cisco.com
>. mailto:mi...@osl.iu.edu
>. https://svn.open-mpi.org/trac/mtt/changeset/1319
>. mailto:mtt-...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>. mailto:jsquy...@cisco.com
>. mailto:mtt-de...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

> 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1314

[Ethan Mallove]

On Wed, Sep/09/2009 10:30:44AM, Mike Dubman wrote:
>Hey Eytan,
> 
>It seems argv is participating in the following scenarios:
> 
>1. argv should be defined in mtt.ini for every single [Test Run] section
>2. Currently, _argv() is returing un-evaluated argv`s value
>3. _argv() is usually part of "exec=" parameter line of [MPI
>Details], which is evaluated for very test invocation:
> 
>mpiexec @options@ -n _np() _executable() _argv()
> 
>According to analysis above, if argv contains funclets or variables, they
>will get expanded during "exec" line evaluation.

Okay, so we delay the evaluation of "argv" to when "exec" is
evaluated. The error case is:

  argv = _np()

Before fix:

  argv is undefined

After fix:

  argv is the value of _np()

-Ethan

> 
>regards
> 
>Mike
> 
>On Tue, Sep 8, 2009 at 9:10 PM, Ethan Mallove 
>wrote:
> 
>  Mike,
> 
>  What if argv contains a funclet, e.g.,
> 
>  *argv = ()
> 
>  Won't this change prevent it from getting expanded?
> 
>  -Ethan
> 
>  On Tue, Sep/08/2009 09:43:37AM, mi...@osl.iu.edu wrote:
>  > Author: miked
>  > Date: 2009-09-08 09:43:37 EDT (Tue, 08 Sep 2009)
>  > New Revision: 1314
>  > URL: https://svn.open-mpi.org/trac/mtt/changeset/1314
>  >
>  > Log:
>  > fix:
>  >
>  > _np() can return incorrect value if used inside argv, here is a
>  scenario:
>  >
>  > This behavior can be explained in next words as evaluation _test()
>  > returns uninitialized $MTT::Test::Run::test_np that is initialized
>  later in _run_one_np function.
>  >
>  > As a result using
>  > $MTT::Test::Run::test_argv = $run->{argv};
>  > allows to avoid damaging $MTT::Test::Run::test_argv *variable on
>  current step but evaluation of _np() is done with whole
>  command_line.
>  >
>  >
>  > Text files modified:
>  > * *trunk/lib/MTT/Test/RunEngine.pm | * * 2 +-
>  > * *1 files changed, 1 insertions(+), 1 deletions(-)
>  >
>  > Modified: trunk/lib/MTT/Test/RunEngine.pm
>  >
>  
> ==
>  > --- trunk/lib/MTT/Test/RunEngine.pm * (original)
>  > +++ trunk/lib/MTT/Test/RunEngine.pm * 2009-09-08 09:43:37 EDT (Tue, 08
>  Sep 2009)
>  > @@ -191,7 +191,7 @@
>  > * * * * *$MTT::Test::Run::test_executable_abspath = $test_exe_abs;
>  > * * * * *$MTT::Test::Run::test_executable_basename =
>  $test_exe_basename;
>  >
>  > - * * * *$MTT::Test::Run::test_argv =
>  MTT::Values::EvaluateString($run->{argv}, $ini, $test_run_full_name);
>  > + * * * *$MTT::Test::Run::test_argv = $run->{argv};
>  > * * * * *my $all_np = MTT::Values::EvaluateString($run->{np}, $ini,
>  $test_run_full_name);
>  >
>  > * * * * *my $save_run_mpi_details = $MTT::Test::Run::mpi_details;
>  > ___
>  > mtt-svn mailing list
>  > mtt-...@open-mpi.org
>  > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>  ___
>  mtt-devel mailing list
>  mtt-de...@open-mpi.org
>  http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> References
> 
>Visible links
>. mailto:ethan.mall...@sun.com
>. mailto:mi...@osl.iu.edu
>. https://svn.open-mpi.org/trac/mtt/changeset/1314
>. mailto:mtt-...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>. mailto:mtt-de...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] [MTT svn] svn:mtt-svn r1314

[Mike Dubman]

Hey Eytan,

It seems argv is participating in the following scenarios:


1. argv should be defined in mtt.ini for every single [Test Run] section
2. Currently, _argv() is returing un-evaluated argv`s value
3. _argv() is usually part of "exec=" parameter line of [MPI Details],
which is evaluated for very test invocation:

mpiexec @options@ -n _np() _executable() _argv()


According to analysis above, if argv contains funclets or variables, they
will get expanded during "exec" line evaluation.

regards

Mike

On Tue, Sep 8, 2009 at 9:10 PM, Ethan Mallove  wrote:

> Mike,
>
> What if argv contains a funclet, e.g.,
>
>  argv = ()
>
> Won't this change prevent it from getting expanded?
>
> -Ethan
>
>
> On Tue, Sep/08/2009 09:43:37AM, mi...@osl.iu.edu wrote:
> > Author: miked
> > Date: 2009-09-08 09:43:37 EDT (Tue, 08 Sep 2009)
> > New Revision: 1314
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1314
> >
> > Log:
> > fix:
> >
> > _np() can return incorrect value if used inside argv, here is a
> scenario:
> >
> > This behavior can be explained in next words as evaluation _test()
> > returns uninitialized $MTT::Test::Run::test_np that is initialized later
> in _run_one_np function.
> >
> > As a result using
> > $MTT::Test::Run::test_argv = $run->{argv};
> > allows to avoid damaging $MTT::Test::Run::test_argv  variable on current
> step but evaluation of _np() is done with whole command_line.
> >
> >
> > Text files modified:
> >trunk/lib/MTT/Test/RunEngine.pm | 2 +-
> >1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/lib/MTT/Test/RunEngine.pm
> >
> ==
> > --- trunk/lib/MTT/Test/RunEngine.pm   (original)
> > +++ trunk/lib/MTT/Test/RunEngine.pm   2009-09-08 09:43:37 EDT (Tue, 08
> Sep 2009)
> > @@ -191,7 +191,7 @@
> >  $MTT::Test::Run::test_executable_abspath = $test_exe_abs;
> >  $MTT::Test::Run::test_executable_basename = $test_exe_basename;
> >
> > -$MTT::Test::Run::test_argv =
> MTT::Values::EvaluateString($run->{argv}, $ini, $test_run_full_name);
> > +$MTT::Test::Run::test_argv = $run->{argv};
> >  my $all_np = MTT::Values::EvaluateString($run->{np}, $ini,
> $test_run_full_name);
> >
> >  my $save_run_mpi_details = $MTT::Test::Run::mpi_details;
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] [MTT svn] svn:mtt-svn r1314

[Ethan Mallove]

Mike,

What if argv contains a funclet, e.g.,

  argv = ()

Won't this change prevent it from getting expanded?

-Ethan


On Tue, Sep/08/2009 09:43:37AM, mi...@osl.iu.edu wrote:
> Author: miked
> Date: 2009-09-08 09:43:37 EDT (Tue, 08 Sep 2009)
> New Revision: 1314
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1314
> 
> Log:
> fix:
> 
> _np() can return incorrect value if used inside argv, here is a scenario:
> 
> This behavior can be explained in next words as evaluation _test()
> returns uninitialized $MTT::Test::Run::test_np that is initialized later in 
> _run_one_np function.
> 
> As a result using
> $MTT::Test::Run::test_argv = $run->{argv};
> allows to avoid damaging $MTT::Test::Run::test_argv  variable on current step 
> but evaluation of _np() is done with whole command_line.
> 
> 
> Text files modified: 
>trunk/lib/MTT/Test/RunEngine.pm | 2 +- 
>  
>1 files changed, 1 insertions(+), 1 deletions(-)
> 
> Modified: trunk/lib/MTT/Test/RunEngine.pm
> ==
> --- trunk/lib/MTT/Test/RunEngine.pm   (original)
> +++ trunk/lib/MTT/Test/RunEngine.pm   2009-09-08 09:43:37 EDT (Tue, 08 Sep 
> 2009)
> @@ -191,7 +191,7 @@
>  $MTT::Test::Run::test_executable_abspath = $test_exe_abs;
>  $MTT::Test::Run::test_executable_basename = $test_exe_basename;
>  
> -$MTT::Test::Run::test_argv = 
> MTT::Values::EvaluateString($run->{argv}, $ini, $test_run_full_name);
> +$MTT::Test::Run::test_argv = $run->{argv};
>  my $all_np = MTT::Values::EvaluateString($run->{np}, $ini, 
> $test_run_full_name);
>  
>  my $save_run_mpi_details = $MTT::Test::Run::mpi_details;
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn

Re: [MTT devel] [MTT svn] svn:mtt-svn r1306

[Ethan Mallove]

On Tue, Aug/11/2009 02:53:50PM, Mike Dubman wrote:
>Hey Jeff,
> 
>This code acts as a pre-processor during loading of ini file into mtt.
>It replaces builtin vars %VAR% with their values, for example:
> 
>...
>[Test run: trivial]
>my_sect_name=%INI_SECTION_NAME%
>...
> 
>%INI_SECTION_NAME% get replaced with real value. (trivial)
> 
>it is useful in the following situation:
> 
>...
>[test run: trivial]
>#param=_run_name()
>param=%INI_SECTION_NAME%
>...
> 
>when "param" was accessed from Reporter context, test_run_name() will
>return undef, but real value if %INI_SECTION_NAME% is used!

Doesn't _simple_section() do the same thing as
%INI_SECTION_NAME%? There are a couple predefined INI vars already,
but they use @VAR@ syntax:

  @INI_NAME@
  @PROGRAM_NAME@

The predefined vars are for strings that can't be known via the
Config::IniFiles module (e.g., the full path to the INI file and
client/mtt).

Could you add INI_SECTION_NAME to InsertINIPredefines, and use the
@VAR@ syntax?

-Ethan

> 
>regards
> 
>Mike
> 
>On Tue, Aug 11, 2009 at 2:03 PM, Jeff Squyres  wrote:
> 
>  Mike --
> 
>  Can you explain what this does?
> 
>  On Aug 11, 2009, at 4:28 AM,  wrote:
> 
>Author: miked
>Date: 2009-08-11 04:28:03 EDT (Tue, 11 Aug 2009)
>New Revision: 1306
>URL: https://svn.open-mpi.org/trac/mtt/changeset/1306
> 
>Log:
>added poor-man-inifile-preprocessor
>Text files modified:
>* trunk/client/mtt * * | * * 3 +++
>* trunk/lib/MTT/INI.pm | * *24 
>* 2 files changed, 27 insertions(+), 0 deletions(-)
> 
>Modified: trunk/client/mtt
>=
>=
>=
>=
>=
>=
>=
>=
>==
>--- trunk/client/mtt * *(original)
>+++ trunk/client/mtt * *2009-08-11 04:28:03 EDT (Tue, 11 Aug 2009)
>@@ -652,6 +652,9 @@
>* * * * # Expand all the "include_section" parameters
>* * * * $unfiltered = MTT::INI::ExpandIncludeSections($unfiltered);
> 
>+ * * * *# Expand all the "%PREDEFINED_VARS%" parameters
>+ * * * *$unfiltered = MTT::INI::ExpandPredefinedVars($unfiltered);
>+
>* * * * # Keep an unfiltered version of the ini file for error
>checking
>* * * * my $filtered = dclone($unfiltered);
> 
>Modified: trunk/lib/MTT/INI.pm
>
> ==
>--- trunk/lib/MTT/INI.pm * * * *(original)
>+++ trunk/lib/MTT/INI.pm * * * *2009-08-11 04:28:03 EDT (Tue, 11 Aug
>2009)
>@@ -275,6 +275,30 @@
>* * return $ini;
>*}
> 
>+sub ExpandPredefinedVars {
>+ * *my($ini) = @_;
>+
>+ * *foreach my $section ($ini->Sections) {
>+ * * * * * * * foreach my $parameter ($ini->Parameters($section)) {
>+ * * * * * * * * * * * my $val = $ini->val($section, $parameter);
>+ * * * * * * * * * * * if ( $val =~ /%INI_SECTION_NAME%/i ) {
>+ * * * * * * * * * * * * * * * my $sect = $section;
>+ * * * * * * * * * * * * * * * $sect =~ s/test run://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/test build://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/test get://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/mpi get://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/mpi install://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/mpi details://gi;
>+ * * * * * * * * * * * * * * * $sect =~ s/reporter://gi;
>+ * * * * * * * * * * * * * * * $val =~ s/%INI_SECTION_NAME%/$sect/g;
>+ * * * * * * * * * * * * * * * $ini->delval($section, $parameter);
>+ * * * * * * * * * * * * * * * $ini->newval($section, $parameter,
>$val);
>+ * * * * * * * * * * * }
>+ * * * * * * * }
>+ * *}
>+ * *return $ini;
>+}
>+
>*# Worker subroutine for recursive ExpandIncludeSections
>*sub _expand_include_sections {
>* * my($ini, $section) = @_;
>___
>mtt-svn mailing list
>mtt-...@open-mpi.org
>http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
>  --
>  Jeff Squyres
>  jsquy...@cisco.com
> 
>  ___
>  mtt-devel mailing list
>  mtt-de...@open-mpi.org
>  http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> References
> 
>Visible links
>. mailto:jsquy...@cisco.com
>. mailto:mi...@osl.iu.edu
>. https://svn.open-mpi.org/trac/mtt/changeset/1306
>. mailto:mtt-...@open-mpi.org
>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>. 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1306

[Mike Dubman]

Hey Jeff,

This code acts as a pre-processor during loading of ini file into mtt.
It replaces builtin vars %VAR% with their values, for example:

...
[Test run: trivial]
my_sect_name=%INI_SECTION_NAME%
...

%INI_SECTION_NAME% get replaced with real value. (trivial)



it is useful in the following situation:

...
[test run: trivial]
#param=_run_name()
param=%INI_SECTION_NAME%
...

when "param" was accessed from Reporter context, test_run_name() will return
undef, but real value if %INI_SECTION_NAME% is used!



regards

Mike

On Tue, Aug 11, 2009 at 2:03 PM, Jeff Squyres  wrote:

> Mike --
>
> Can you explain what this does?
>
> On Aug 11, 2009, at 4:28 AM,  wrote:
>
>  Author: miked
>> Date: 2009-08-11 04:28:03 EDT (Tue, 11 Aug 2009)
>> New Revision: 1306
>> URL: https://svn.open-mpi.org/trac/mtt/changeset/1306
>>
>> Log:
>> added poor-man-inifile-preprocessor
>> Text files modified:
>>   trunk/client/mtt | 3 +++
>>   trunk/lib/MTT/INI.pm |24 
>>   2 files changed, 27 insertions(+), 0 deletions(-)
>>
>> Modified: trunk/client/mtt
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> =
>> ==
>> --- trunk/client/mtt(original)
>> +++ trunk/client/mtt2009-08-11 04:28:03 EDT (Tue, 11 Aug 2009)
>> @@ -652,6 +652,9 @@
>> # Expand all the "include_section" parameters
>> $unfiltered = MTT::INI::ExpandIncludeSections($unfiltered);
>>
>> +# Expand all the "%PREDEFINED_VARS%" parameters
>> +$unfiltered = MTT::INI::ExpandPredefinedVars($unfiltered);
>> +
>> # Keep an unfiltered version of the ini file for error checking
>> my $filtered = dclone($unfiltered);
>>
>>
>> Modified: trunk/lib/MTT/INI.pm
>>
>> ==
>> --- trunk/lib/MTT/INI.pm(original)
>> +++ trunk/lib/MTT/INI.pm2009-08-11 04:28:03 EDT (Tue, 11 Aug 2009)
>> @@ -275,6 +275,30 @@
>> return $ini;
>>  }
>>
>> +sub ExpandPredefinedVars {
>> +my($ini) = @_;
>> +
>> +foreach my $section ($ini->Sections) {
>> +   foreach my $parameter ($ini->Parameters($section)) {
>> +   my $val = $ini->val($section, $parameter);
>> +   if ( $val =~ /%INI_SECTION_NAME%/i ) {
>> +   my $sect = $section;
>> +   $sect =~ s/test run://gi;
>> +   $sect =~ s/test build://gi;
>> +   $sect =~ s/test get://gi;
>> +   $sect =~ s/mpi get://gi;
>> +   $sect =~ s/mpi install://gi;
>> +   $sect =~ s/mpi details://gi;
>> +   $sect =~ s/reporter://gi;
>> +   $val =~ s/%INI_SECTION_NAME%/$sect/g;
>> +   $ini->delval($section, $parameter);
>> +   $ini->newval($section, $parameter, $val);
>> +   }
>> +   }
>> +}
>> +return $ini;
>> +}
>> +
>>  # Worker subroutine for recursive ExpandIncludeSections
>>  sub _expand_include_sections {
>> my($ini, $section) = @_;
>> ___
>> mtt-svn mailing list
>> mtt-...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>>
>>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] MTT email timeout notification feature

[Ethan Mallove]

In r1300, I incorporated your notes, except for #2 and #3. I dug
around for a CPAN module for stack traces, but couldn't find anything.
Maybe MTT could contain a stripped down version of GDB for the sole
purpose of gathering stack traces? Or is there some other open source
"stack trace grabber" tool out there?

-Ethan

On Mon, Jun/22/2009 07:37:16AM, Jeff Squyres wrote:
> Actually, I think this would be fine for the trunk.  Some random notes:
>
> 1. It might be nice to move this logic out of the docommand sub itself and 
> into its own sub.
> 2. it would also be good to generalize the ps and gdb commands for systems 
> where those variants are not relevant
> 3. it might even be good to generally develop the backtrace functionality 
> overall -- backtraces would be really good to capture in the database...
> 4. how about having a[n optional] timeout with the sentinel file?  that is, 
> it'll send a mail, then wait another timeout (e.g., 1 hour) and if the 
> sentinel file still exists, mtt will remove the file and keep going
>
>
> On Jun 19, 2009, at 2:47 PM, Ethan Mallove wrote:
>
>> Folks,
>>
>> I came up with a feature, which does not seem quite appropriate to go
>> into the MTT trunk, but is still possibly useful for someone other
>> than me. I have posted a note about it on the MTT wiki:
>>
>>   http://svn.open-mpi.org/trac/mtt/wiki/EmailTimeoutNotification
>>
>> Here's the text of the Wiki page:
>>
>> We (Sun) were trying to track down a hang in an MPI test that we were
>> seeing in our MTT runs which was difficult to reproduce manually. The
>> problem is that MTT kills the hanging process before a developer has a
>> chance to investigate the issue. To address this, I patched an MTT
>> client (see attached patch file) to send out a notification email
>> containing an mpirun command line and GDB back trace for the hanging
>> test. A predefined sentinel file is touched, which can later be
>> removed to force MTT to move on and continue testing. Here are the INI
>> parameters to activate the timeout email notification:
>>
>>  * {{{docommand_timeout_sentinel_file}}}
>>  * {{{docommand_timeout_email_recipient}}}
>>
>> Example usage:
>>
>> {{{
>> $ client/mtt --scratch /foo/bar --file foo.ini
>>   
>> docommand_timeout_sentinel_file=/tmp/mtt-timeout-sentinel-file-\_string\(10\)
>>   
>> docommand_timeout_email_recipient=fred.flints...@sun.com,barney.rub...@sun.com
>> }}}
>>
>> -Ethan
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>
>
> -- 
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] MTT

[Jeff Squyres]

Shiqing and I talked off-list and fixed the problem.  I committed to  
both trunk and the v3.0 branch.



On Apr 30, 2009, at 4:10 AM, Shiqing Fan wrote:


Hi Jeff,

Yes, I just found out that "-o" option doesn't exist on OS X. But
problem is we don't have client or whatami on Cygwin, are they Linux
shell commands?

Anyway, I figured out another way to get the system type with Perl,  
just
"my $sys_type=$^O" will give us the same result as using "uname -o".  
I'm
not a Perl expert, so I don't know how it works, but here is a link  
from

Perl dev-reference:
http://perl.devquickref.com/perl-get-current-operating-system.html

Could this be a solution?


Regards,
Shiqing


Jeff Squyres wrote:
> Shiqing --
>
> It looks like the "uname -o" that was added into OMPI.pm is
> problematic on OS X.
>
> What does client/whatami/whatami return on the platforms that you  
care

> about?  I.e., can we re-write this chunk of code in
> MPI/Install/OMPI.pm to use whatami output:
>
> my $sys_type=`uname -o`;
> if(($sys_type =~ /cygwin/i || $sys_type =~ /msys/i) &&
> $config->{compiler_name} eq "microsoft") {
> $install = MTT::Common::Cmake::Install($gnu);
> } else {
> $install = MTT::Common::GNU_Install::Install($gnu);
> }
>


--
--
Shiqing Fan  http://www.hlrs.de/people/fan
High Performance Computing   Tel.: +49 711 685 87234
  Center Stuttgart (HLRS)Fax.: +49 711 685 65832
Address:Allmandring 30   email: f...@hlrs.de
70569 Stuttgart




--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273(Analyze/Performance plug-ins)

[Jeff Squyres]

Let's have a teleconf/meeting/whatever and explore the possibility of  
the google data store.  That might make the complexity of the database  
schema issues we had for arbitrary performance data be significantly  
easier...?


(perhaps I'm being too optimistic)


On Mar 20, 2009, at 10:33 AM, Josh Hursey wrote:



A while back we designed what needed to happen on the database side
to support arbitrary performance data (we were thinking about Skampi
at the time, I believe). The only documentation that I could find of
this discussion is in the DB schema diagram:
   https://svn.open-mpi.org/trac/mtt/browser/trunk/docs/sql-schema-
v3.pdf

On the right-hand side, we currently implement the
'latency_bandwidth' table. We would like to move to the
'performance_data' table which would encode any performance data that
can be represented in a 3 dimensional space

If someone wanted to go down the path of converting the performance
representation, I can probably dig out some more notes on what we
were thinking at the time of this design.

Best,
Josh

On Mar 20, 2009, at 9:49 AM, Ethan Mallove wrote:

> On Thu, Mar/19/2009 09:17:29PM, Mike Dubman wrote:
>>Hello Eithan,
>>Thanks for info, will refactor it.
>>
>>from http://www.netlib.org/benchmark/hpl/
>>
>>...
>>HPL is a software package that solves a (random) dense linear
>>system in double precision (64 bits) arithmetic on
>>distributed-memory computers. It can thus be regarded as a
>>portable as well as freely available implementation of the High
>>Performance Computing Linpack Benchmark.
>>
>>The HPL package provides a testing and timing program to  
quantify
>>the accuracy of the obtained solution as well as the time it  
took

>>to compute it
>>...
>>
>>Where do you think is a good place to keep parsers for other  
then

>>lat/bw based mpi benchmarks? I think we can have a collection of
>>such parsers in the mtt and at some point we can enhance mtt
>>reports with other metrics.
>>
>>What do you think?
>
> Sounds like a nice enhancement.  I think just assigning this parser
> its own "test_type" will be helpful for now (and I have no  
thoughts on
> what to call it).  When the server-side submit PHP script is ready  
to

> handle non-latency/bandwidth test data, we can split out the
> client-side Analyze/Performance directory.
>
> -Ethan
>
>
>>
>>On Thu, Mar 19, 2009 at 8:22 PM, Ethan Mallove
>> 
>>wrote:
>>
>>  Hi Mike,
>>
>>  Is HPL a latency and/or bandwidth performance test? *All the
>> Analyze
>>  plug-ins in lib/MTT/Test/Analyze/Performance are for latency/
>> bandwidth
>>  tests, which means they can then be rendered as graphs in the
>> MTT
>>  Reporter. *All of these plug-ins are required to output at
>> least one
>>  of the following:
>>
>>  *latency_avg
>>  *latency_min
>>  *latency_max
>>  *bandwidth_avg
>>  *bandwidth_min
>>  *bandwidth_max
>>
>>  They all contain this:
>>
>>  *$report->{test_type} = 'latency_bandwidth';
>>
>>  HPL.pm should have a line like this somewhere:
>>
>>  *$report->{test_type} = 'tv_gflops';
>>
>>  Maybe HPL.pm could go into a different directory or have a
>> comment
>>  somewhere to clear up this confusion.
>>
>>  Regards,
>>  Ethan
>>
>>  On Thu, Mar/19/2009 02:11:05AM, mi...@osl.iu.edu wrote:
>>> Author: miked
>>> Date: 2009-03-19 02:11:04 EDT (Thu, 19 Mar 2009)
>>> New Revision: 1273
>>> URL: https://svn.open-mpi.org/trac/mtt/changeset/1273
>>>
>>> Log:
>>> HPL analyzer added
>>>
>>> Added:
>>> * *trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
>>>
>>> Added: trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
>>>
>>
>>  
=

>> =
>>> --- (empty file)
>>> +++ trunk/lib/MTT/Test/Analyze/Performance/HPL.pm * * 2009-03-19
>>  02:11:04 EDT (Thu, 19 Mar 2009)
>>> @@ -0,0 +1,63 @@
>>> +#!/usr/bin/env perl
>>> +#
>>> +# Copyright (c) 2006-2007 Sun Microsystems, Inc. *All rights
>>  reserved.
>>> +# Copyright (c) 2007 * * *Voltaire *All rights reserved.
>>> +# $COPYRIGHT$
>>> +#
>>> +# Additional copyrights may follow
>>> +#
>>> +# $HEADER$
>>> +#
>>> +
>>> +package MTT::Test::Analyze::Performance::HPL;
>>> +use strict;
>>> +use Data::Dumper;
>>> +#use MTT::Messages;
>>> +
>>> +# Process the result_stdout emitted from one of hpl tests
>>> +sub Analyze {
>>> +
>>> + * *my($result_stdout) = @_;
>>> + * *my $report;
>>> + * *my(@t_v,
>>> + * * * @time,
>>> + * * * @gflops);
>>> +
>>> +$report->{test_name}="HPL";
>>> + * *my @lines = split(/\n|\r/, $result_stdout);
>>> + * *# Sample result_stdout:
>>> +#- The matrix A is randomly generated for each test.
>>> +#- The following scaled residual check will be computed:
>>> +# * * *||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||
>>> _oo )
>>  * N )
>>> +#- The relative machine precision (eps) is taken to be * * * * *

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273 (Analyze/Performance plug-ins)

[Josh Hursey]

A while back we designed what needed to happen on the database side  
to support arbitrary performance data (we were thinking about Skampi  
at the time, I believe). The only documentation that I could find of  
this discussion is in the DB schema diagram:
  https://svn.open-mpi.org/trac/mtt/browser/trunk/docs/sql-schema- 
v3.pdf


On the right-hand side, we currently implement the  
'latency_bandwidth' table. We would like to move to the  
'performance_data' table which would encode any performance data that  
can be represented in a 3 dimensional space


If someone wanted to go down the path of converting the performance  
representation, I can probably dig out some more notes on what we  
were thinking at the time of this design.


Best,
Josh

On Mar 20, 2009, at 9:49 AM, Ethan Mallove wrote:


On Thu, Mar/19/2009 09:17:29PM, Mike Dubman wrote:

   Hello Eithan,
   Thanks for info, will refactor it.

   from http://www.netlib.org/benchmark/hpl/

   ...
   HPL is a software package that solves a (random) dense linear
   system in double precision (64 bits) arithmetic on
   distributed-memory computers. It can thus be regarded as a
   portable as well as freely available implementation of the High
   Performance Computing Linpack Benchmark.

   The HPL package provides a testing and timing program to quantify
   the accuracy of the obtained solution as well as the time it took
   to compute it
   ...

   Where do you think is a good place to keep parsers for other then
   lat/bw based mpi benchmarks? I think we can have a collection of
   such parsers in the mtt and at some point we can enhance mtt
   reports with other metrics.

   What do you think?


Sounds like a nice enhancement.  I think just assigning this parser
its own "test_type" will be helpful for now (and I have no thoughts on
what to call it).  When the server-side submit PHP script is ready to
handle non-latency/bandwidth test data, we can split out the
client-side Analyze/Performance directory.

-Ethan




   On Thu, Mar 19, 2009 at 8:22 PM, Ethan Mallove  


   wrote:

 Hi Mike,

 Is HPL a latency and/or bandwidth performance test? *All the  
Analyze
 plug-ins in lib/MTT/Test/Analyze/Performance are for latency/ 
bandwidth
 tests, which means they can then be rendered as graphs in the  
MTT
 Reporter. *All of these plug-ins are required to output at  
least one

 of the following:

 *latency_avg
 *latency_min
 *latency_max
 *bandwidth_avg
 *bandwidth_min
 *bandwidth_max

 They all contain this:

 *$report->{test_type} = 'latency_bandwidth';

 HPL.pm should have a line like this somewhere:

 *$report->{test_type} = 'tv_gflops';

 Maybe HPL.pm could go into a different directory or have a  
comment

 somewhere to clear up this confusion.

 Regards,
 Ethan

 On Thu, Mar/19/2009 02:11:05AM, mi...@osl.iu.edu wrote:

Author: miked
Date: 2009-03-19 02:11:04 EDT (Thu, 19 Mar 2009)
New Revision: 1273
URL: https://svn.open-mpi.org/trac/mtt/changeset/1273

Log:
HPL analyzer added

Added:
* *trunk/lib/MTT/Test/Analyze/Performance/HPL.pm

Added: trunk/lib/MTT/Test/Analyze/Performance/HPL.pm

  
= 
=

--- (empty file)
+++ trunk/lib/MTT/Test/Analyze/Performance/HPL.pm * * 2009-03-19

 02:11:04 EDT (Thu, 19 Mar 2009)

@@ -0,0 +1,63 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2006-2007 Sun Microsystems, Inc. *All rights

 reserved.

+# Copyright (c) 2007 * * *Voltaire *All rights reserved.
+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$
+#
+
+package MTT::Test::Analyze::Performance::HPL;
+use strict;
+use Data::Dumper;
+#use MTT::Messages;
+
+# Process the result_stdout emitted from one of hpl tests
+sub Analyze {
+
+ * *my($result_stdout) = @_;
+ * *my $report;
+ * *my(@t_v,
+ * * * @time,
+ * * * @gflops);
+
+$report->{test_name}="HPL";
+ * *my @lines = split(/\n|\r/, $result_stdout);
+ * *# Sample result_stdout:
+#- The matrix A is randomly generated for each test.
+#- The following scaled residual check will be computed:
+# * * *||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b || 
_oo )

 * N )
+#- The relative machine precision (eps) is taken to be * * * * *  
* *

 1.110223e-16
+#- Computational tests pass if scaled residuals are less than *  
* * *

 * * * *16.0


  
+#=== 
=
+#T/V * * * * * * * *N * *NB * * P * * Q * * * * * * * Time * * *  
* *

 * * * Gflops


  
+#--- 
-
+#WR00L2L2 * * * 29184 * 128 * * 2 * * 4 * * * * * 15596.86 * * *  
* *

 * *1.063e+00


  
+#--- 
-

+#||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)= * * * *0.0008986

 .. PASSED


  

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273 (Analyze/Performance plug-ins)

[Ethan Mallove]

On Thu, Mar/19/2009 09:17:29PM, Mike Dubman wrote:
>Hello Eithan,
>Thanks for info, will refactor it.
> 
>from http://www.netlib.org/benchmark/hpl/
> 
>...
>HPL is a software package that solves a (random) dense linear
>system in double precision (64 bits) arithmetic on
>distributed-memory computers. It can thus be regarded as a
>portable as well as freely available implementation of the High
>Performance Computing Linpack Benchmark.
> 
>The HPL package provides a testing and timing program to quantify
>the accuracy of the obtained solution as well as the time it took
>to compute it
>...
> 
>Where do you think is a good place to keep parsers for other then
>lat/bw based mpi benchmarks? I think we can have a collection of
>such parsers in the mtt and at some point we can enhance mtt
>reports with other metrics.
> 
>What do you think?

Sounds like a nice enhancement.  I think just assigning this parser
its own "test_type" will be helpful for now (and I have no thoughts on
what to call it).  When the server-side submit PHP script is ready to
handle non-latency/bandwidth test data, we can split out the
client-side Analyze/Performance directory.

-Ethan


> 
>On Thu, Mar 19, 2009 at 8:22 PM, Ethan Mallove 
>wrote:
> 
>  Hi Mike,
> 
>  Is HPL a latency and/or bandwidth performance test? *All the Analyze
>  plug-ins in lib/MTT/Test/Analyze/Performance are for latency/bandwidth
>  tests, which means they can then be rendered as graphs in the MTT
>  Reporter. *All of these plug-ins are required to output at least one
>  of the following:
> 
>  *latency_avg
>  *latency_min
>  *latency_max
>  *bandwidth_avg
>  *bandwidth_min
>  *bandwidth_max
> 
>  They all contain this:
> 
>  *$report->{test_type} = 'latency_bandwidth';
> 
>  HPL.pm should have a line like this somewhere:
> 
>  *$report->{test_type} = 'tv_gflops';
> 
>  Maybe HPL.pm could go into a different directory or have a comment
>  somewhere to clear up this confusion.
> 
>  Regards,
>  Ethan
> 
>  On Thu, Mar/19/2009 02:11:05AM, mi...@osl.iu.edu wrote:
>  > Author: miked
>  > Date: 2009-03-19 02:11:04 EDT (Thu, 19 Mar 2009)
>  > New Revision: 1273
>  > URL: https://svn.open-mpi.org/trac/mtt/changeset/1273
>  >
>  > Log:
>  > HPL analyzer added
>  >
>  > Added:
>  > * *trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
>  >
>  > Added: trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
>  >
>  
> ==
>  > --- (empty file)
>  > +++ trunk/lib/MTT/Test/Analyze/Performance/HPL.pm * * 2009-03-19
>  02:11:04 EDT (Thu, 19 Mar 2009)
>  > @@ -0,0 +1,63 @@
>  > +#!/usr/bin/env perl
>  > +#
>  > +# Copyright (c) 2006-2007 Sun Microsystems, Inc. *All rights
>  reserved.
>  > +# Copyright (c) 2007 * * *Voltaire *All rights reserved.
>  > +# $COPYRIGHT$
>  > +#
>  > +# Additional copyrights may follow
>  > +#
>  > +# $HEADER$
>  > +#
>  > +
>  > +package MTT::Test::Analyze::Performance::HPL;
>  > +use strict;
>  > +use Data::Dumper;
>  > +#use MTT::Messages;
>  > +
>  > +# Process the result_stdout emitted from one of hpl tests
>  > +sub Analyze {
>  > +
>  > + * *my($result_stdout) = @_;
>  > + * *my $report;
>  > + * *my(@t_v,
>  > + * * * @time,
>  > + * * * @gflops);
>  > +
>  > +$report->{test_name}="HPL";
>  > + * *my @lines = split(/\n|\r/, $result_stdout);
>  > + * *# Sample result_stdout:
>  > +#- The matrix A is randomly generated for each test.
>  > +#- The following scaled residual check will be computed:
>  > +# * * *||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo )
>  * N )
>  > +#- The relative machine precision (eps) is taken to be * * * * * * *
>  1.110223e-16
>  > +#- Computational tests pass if scaled residuals are less than * * * *
>  * * * *16.0
>  >
>  
> +#
>  > +#T/V * * * * * * * *N * *NB * * P * * Q * * * * * * * Time * * * * *
>  * * * Gflops
>  >
>  
> +#
>  > +#WR00L2L2 * * * 29184 * 128 * * 2 * * 4 * * * * * 15596.86 * * * * *
>  * *1.063e+00
>  >
>  
> +#
>  > +#||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)= * * * *0.0008986
>  .. PASSED
>  >
>  
> +#
>  > +#T/V * * * * * * * *N * *NB * * P * * Q * * * * * * * Time * * * * *
>  * * * Gflops
>  >
>  
> 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273 (Analyze/Performance plug-ins)

[Mike Dubman]

Hello Eithan,
Thanks for info, will refactor it.

from http://www.netlib.org/benchmark/hpl/

...
*HPL* is a software package that solves a (random) dense linear system in
double precision (64 bits) arithmetic on distributed-memory computers. It
can thus be regarded as a portable as well as freely available
implementation of the High Performance Computing Linpack Benchmark.

The HPL package provides a testing and timing program to quantify the *
accuracy* of the obtained solution as well as the time it took to compute it
...


Where do you think is a good place to keep parsers for other then lat/bw
based mpi benchmarks?
I think we can have a collection of such parsers in the mtt and at some
point we can enhance mtt reports with other metrics.

What do you think?


On Thu, Mar 19, 2009 at 8:22 PM, Ethan Mallove wrote:

> Hi Mike,
>
> Is HPL a latency and/or bandwidth performance test?  All the Analyze
> plug-ins in lib/MTT/Test/Analyze/Performance are for latency/bandwidth
> tests, which means they can then be rendered as graphs in the MTT
> Reporter.  All of these plug-ins are required to output at least one
> of the following:
>
>  latency_avg
>  latency_min
>  latency_max
>  bandwidth_avg
>  bandwidth_min
>  bandwidth_max
>
> They all contain this:
>
>  $report->{test_type} = 'latency_bandwidth';
>
> HPL.pm should have a line like this somewhere:
>
>  $report->{test_type} = 'tv_gflops';
>
> Maybe HPL.pm could go into a different directory or have a comment
> somewhere to clear up this confusion.
>
> Regards,
> Ethan
>
>
> On Thu, Mar/19/2009 02:11:05AM, mi...@osl.iu.edu wrote:
> > Author: miked
> > Date: 2009-03-19 02:11:04 EDT (Thu, 19 Mar 2009)
> > New Revision: 1273
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1273
> >
> > Log:
> > HPL analyzer added
> >
> > Added:
> >trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
> >
> > Added: trunk/lib/MTT/Test/Analyze/Performance/HPL.pm
> >
> ==
> > --- (empty file)
> > +++ trunk/lib/MTT/Test/Analyze/Performance/HPL.pm 2009-03-19 02:11:04
> EDT (Thu, 19 Mar 2009)
> > @@ -0,0 +1,63 @@
> > +#!/usr/bin/env perl
> > +#
> > +# Copyright (c) 2006-2007 Sun Microsystems, Inc.  All rights reserved.
> > +# Copyright (c) 2007  Voltaire  All rights reserved.
> > +# $COPYRIGHT$
> > +#
> > +# Additional copyrights may follow
> > +#
> > +# $HEADER$
> > +#
> > +
> > +package MTT::Test::Analyze::Performance::HPL;
> > +use strict;
> > +use Data::Dumper;
> > +#use MTT::Messages;
> > +
> > +# Process the result_stdout emitted from one of hpl tests
> > +sub Analyze {
> > +
> > +my($result_stdout) = @_;
> > +my $report;
> > +my(@t_v,
> > +   @time,
> > +   @gflops);
> > +
> > +$report->{test_name}="HPL";
> > +my @lines = split(/\n|\r/, $result_stdout);
> > +# Sample result_stdout:
> > +#- The matrix A is randomly generated for each test.
> > +#- The following scaled residual check will be computed:
> > +#  ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) *
> N )
> > +#- The relative machine precision (eps) is taken to be
> 1.110223e-16
> > +#- Computational tests pass if scaled residuals are less than
>  16.0
> >
> +#
> > +#T/VNNB P Q   Time
>   Gflops
> >
> +#
> > +#WR00L2L2   29184   128 2 4   15596.86
>  1.063e+00
> >
> +#
> > +#||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=0.0008986
> .. PASSED
> >
> +#
> > +#T/VNNB P Q   Time
>   Gflops
> >
> +#
> > +#WR00L2L4   29184   128 2 4   15251.81
>  1.087e+00
> > +my $line;
> > +while (defined($line = shift(@lines))) {
> > +#WR00L2L2   29184   128 2 4   15596.86
>1.063e+00
> > +if ($line =~
> m/^(\S+)\s+\d+\s+\d+\s+\d+\s+\d+\s+(\d+[\.\d]+)\s+(\S+)/) {
> > +push(@t_v, $1);
> > +push(@time, $2);
> > +push(@gflops, $3);
> > +}
> > +}
> > +
> > +  # Postgres uses brackets for array insertion
> > +# (see postgresql.org/docs/7.4/interactive/arrays.html)
> > +$report->{tv}   = "{" . join(",", @t_v) . "}";
> > +$report->{time}   = "{" . join(",", @time) . "}";
> > +$report->{gflops}   = "{" . join(",", @gflops) . "}";
> > +return $report;
> > +}
> > +
> > +1;
> > +
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>

Re: [MTT devel] mtt text report oddity

[Jeff Squyres]

On Mar 19, 2009, at 3:19 AM, Mike Dubman wrote:

because the results are rendered in chunks during reporting phase.  
(100 pieces every flush)
This caused same benchmark line to appear more then once in the  
final report.


Ahhh... right.

You can configure the reporter to issue results not by number, but  
for same benchmark at once:


put this in the ini file:

[MTT]
submit_group_results=1


Perfect.  Thanks!

Also, html report is nicer and allows you easy navigation to the  
errors


True, but my HTML files kept getting overwritten by successive submits  
in this case.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt text report oddity

[Mike Dubman]

because the results are rendered in chunks during reporting phase. (100
pieces every flush)
This caused same benchmark line to appear more then once in the final
report.

You can configure the reporter to issue results not by number, but for same
benchmark at once:

put this in the ini file:

[MTT]
submit_group_results=1


Also, html report is nicer and allows you easy navigation to the errors

regards

Mike


2009/3/19 Jeff Squyres 

> I got a fairly odd mtt text report (it's super wide, sorry):
>
> | Test Run| intel | 1.3.1rc5| 00:12| 5|  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:59| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:08| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:51| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:59| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:48| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:10| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:05| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:09| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:25| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:46| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:59| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:23| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:50| 100  | 1|
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:56| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:53| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:22| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 04:21| 100  |  |
> 1|  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 04:12| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:36| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:48| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:47| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 03:08| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:57| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:43| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
> | Test Run| intel | 1.3.1rc5| 02:48| 101  |  |
>  |  | Test_Run-intel-developer-1.3.1rc5.html|
>
> Notice that there are *many* "intel" lines, each with 101 passes.  The only
> difference between them is the times that they ran -- but there's even
> repeats of that.
>
> Do we know why there is so many different lines for the intel test suite?
>
> Did this get changed in the text reporter changes from Voltaire (somewhat)
> recently?
>
> --
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> 

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Hi Jeff,

Thanks a lot.

I'm waiting for reply from the CPAN author.


Cheers,
Shiqing


Jeff Squyres wrote:

Applied.

https://svn.open-mpi.org/trac/mtt/changeset/1272

You should probably push your patch to Filesys::Diskfree upstream to 
the CPAN author:


http://search.cpan.org/~abarclay/Filesys-DiskFree-0.06/DiskFree.pm

(i.e., we wholly imported this module into MTT; we didn't write it)


On Mar 12, 2009, at 9:24 AM, Shiqing Fan wrote:


Hi Jeff,

No problem. I made a tarball for you. :-)


Thanks,
Shiqing

Jeff Squyres wrote:
> This could be an artifact of the new SVN manager system that IU
> installed yesterday; I've sent a mail to the admin asking about it.
> Sorry!  :-(
>
> If you could send me your patch as an attachment, I can commit it (OS
> X Mail.app does weird things with the spacing of inlined patches 
:-( ).

>
>
> On Mar 12, 2009, at 7:00 AM, Shiqing Fan wrote:
>
>> Here is some output of 'svn info':
>>
>> Path: .
>> URL: https://svn.open-mpi.org/svn/mtt/trunk
>> Repository Root: https://svn.open-mpi.org/svn/mtt
>> Repository UUID: 3a00f1f0-e206-0410-aee5-9638c71ae14b
>> Revision: 1271
>> Node Kind: directory
>> Schedule: normal
>> Last Changed Author: emallove
>> Last Changed Rev: 1271
>> Last Changed Date: 2009-02-05 18:22:38 +0100 (Thu, 05 Feb 2009)
>>
>>
>> Is that ok, if someone else commits the patch instead of me? I don't
>> mind actually. :-)
>>
>>
>> Thanks,
>> Shiqing

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

Applied.

https://svn.open-mpi.org/trac/mtt/changeset/1272

You should probably push your patch to Filesys::Diskfree upstream to  
the CPAN author:


http://search.cpan.org/~abarclay/Filesys-DiskFree-0.06/DiskFree.pm

(i.e., we wholly imported this module into MTT; we didn't write it)


On Mar 12, 2009, at 9:24 AM, Shiqing Fan wrote:


Hi Jeff,

No problem. I made a tarball for you. :-)


Thanks,
Shiqing

Jeff Squyres wrote:
> This could be an artifact of the new SVN manager system that IU
> installed yesterday; I've sent a mail to the admin asking about it.
> Sorry!  :-(
>
> If you could send me your patch as an attachment, I can commit it  
(OS
> X Mail.app does weird things with the spacing of inlined patches :- 
( ).

>
>
> On Mar 12, 2009, at 7:00 AM, Shiqing Fan wrote:
>
>> Here is some output of 'svn info':
>>
>> Path: .
>> URL: https://svn.open-mpi.org/svn/mtt/trunk
>> Repository Root: https://svn.open-mpi.org/svn/mtt
>> Repository UUID: 3a00f1f0-e206-0410-aee5-9638c71ae14b
>> Revision: 1271
>> Node Kind: directory
>> Schedule: normal
>> Last Changed Author: emallove
>> Last Changed Rev: 1271
>> Last Changed Date: 2009-02-05 18:22:38 +0100 (Thu, 05 Feb 2009)
>>
>>
>> Is that ok, if someone else commits the patch instead of me? I  
don't

>> mind actually. :-)
>>
>>
>> Thanks,
>> Shiqing





--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Hi Jeff,

No problem. I made a tarball for you. :-)


Thanks,
Shiqing

Jeff Squyres wrote:
This could be an artifact of the new SVN manager system that IU 
installed yesterday; I've sent a mail to the admin asking about it.  
Sorry!  :-(


If you could send me your patch as an attachment, I can commit it (OS 
X Mail.app does weird things with the spacing of inlined patches :-( ).



On Mar 12, 2009, at 7:00 AM, Shiqing Fan wrote:


Here is some output of 'svn info':

Path: .
URL: https://svn.open-mpi.org/svn/mtt/trunk
Repository Root: https://svn.open-mpi.org/svn/mtt
Repository UUID: 3a00f1f0-e206-0410-aee5-9638c71ae14b
Revision: 1271
Node Kind: directory
Schedule: normal
Last Changed Author: emallove
Last Changed Rev: 1271
Last Changed Date: 2009-02-05 18:22:38 +0100 (Thu, 05 Feb 2009)


Is that ok, if someone else commits the patch instead of me? I don't
mind actually. :-)


Thanks,
Shiqing




mtt-windows.tar.gz
Description: GNU Zip compressed data

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Here is some output of 'svn info':

Path: .
URL: https://svn.open-mpi.org/svn/mtt/trunk
Repository Root: https://svn.open-mpi.org/svn/mtt
Repository UUID: 3a00f1f0-e206-0410-aee5-9638c71ae14b
Revision: 1271
Node Kind: directory
Schedule: normal
Last Changed Author: emallove
Last Changed Rev: 1271
Last Changed Date: 2009-02-05 18:22:38 +0100 (Thu, 05 Feb 2009)


Is that ok, if someone else commits the patch instead of me? I don't 
mind actually. :-)



Thanks,
Shiqing


Jeff Squyres wrote:
Did you check out via https?  We only allow commits to all OMPI SVN 
repositories via https.


On Mar 12, 2009, at 5:10 AM, Shiqing Fan wrote:


Hi Ethan and Jeff,

Thanks for all your help. Now it's been fixed and tested.

But it seems that I don't have permission to commit the patch ( I got
403 forbidden ). Any idea?


Thanks,
Shiqing





Index: lib/Filesys/DiskFree.pm
===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,16 @@
'inodes' => "df -Pi",
'format' => "linuxish",
 },
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
+'msys' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
 'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/Cmake.pm
===
--- lib/MTT/Common/Cmake.pm (revision 0)
+++ lib/MTT/Common/Cmake.pm (revision 0)
@@ -0,0 +1,77 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2005-2006 The Trustees of Indiana University.
+# All rights reserved.
+# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
+# Copyright (c) 2009  High Performance Computing Center Stuttgart, 
+# University of Stuttgart.  All rights reserved.
+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$
+#
+
+package MTT::Common::Cmake;
+my $package = ModuleName(__PACKAGE__);
+
+use strict;
+use MTT::Messages;
+use MTT::Values;
+use MTT::Common::Do_step;
+
+#--
+
+# Do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments source_path
+#   [ ] devenv OpenMPI.sln /build
+sub Install {
+my ($config) = @_;
+
+my $x;
+my $result_stdout;
+my $result_stderr;
+
+# Prepare $ret
+my $ret;
+$ret->{test_result} = MTT::Values::FAIL;
+$ret->{exit_status} = 0;
+
+# On windows, do the following steps:
+
+# prepare the windows style prefix.
+# replace '/cygdrive/x/' with 'x:/'
+my $win_prefix = substr ($config->{installdir},10,1) . ":" . substr 
($config->{installdir},11);
+
+# generate Visual Studio solution files
+# use 'script' to redirect MS command output
+$x = MTT::Common::Do_step::do_step($config,
+"script -a -c \"cmake 
$config->{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH=$win_prefix .\" -f 
temp.txt", 
+$config->{merge_stdout_stderr});
+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# compile the whole solution
+$x = MTT::Common::Do_step::do_step($config, "script -a -c \"devenv.com 
*.sln /build debug\" -f temp.txt",
+$config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# install to the prefix dir
+$x = MTT::Common::Do_step::do_step($config, "script -a -c \"devenv.com 
*.sln /project INSTALL.vcproj /build\" -f temp.txt",
+$config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+}
+
+1;
Index: lib/MTT/Common/Do_step.pm
===
--- lib/MTT/Common/Do_step.pm   (revision 0)
+++ lib/MTT/Common/Do_step.pm   (revision 0)
@@ -0,0 +1,139 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2005-2006 The Trustees of Indiana University.
+# All rights reserved.
+# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
+# Copyright (c) 2009  High Performance Computing Center Stuttgart, 
+# University of Stuttgart.  All rights reserved.
+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Hi Jeff,

Yes, I tried also via https and the same account/passwort as I use for 
OMPI repository, but it's not working.



Thanks,
Shiqing


Jeff Squyres wrote:
Did you check out via https?  We only allow commits to all OMPI SVN 
repositories via https.


On Mar 12, 2009, at 5:10 AM, Shiqing Fan wrote:


Hi Ethan and Jeff,

Thanks for all your help. Now it's been fixed and tested.

But it seems that I don't have permission to commit the patch ( I got
403 forbidden ). Any idea?


Thanks,
Shiqing





Ethan Mallove wrote:
> On Wed, Mar/11/2009 03:47:22PM, Jeff Squyres wrote:
>
>> Thanks for your patience!  Yes, this looks good to me with one 
minor nit:

>>
>>
>>> +if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
>>> +$config->{compiler_name} == "microsoft") {
>>>
>> should be
>>
>>
>>> +if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
>>> +$config->{compiler_name} eq "microsoft") {
>>>
>> Josh / Ethan -- got any comments for Shiqing?
>>
>
> I just noticed last night's CYGWIN results. Nice!
>
> Should a regexp (see below) be used in case the sys_type comes back 
as, e.g.,

> "cygwin", "CYGWIN", "cygwin-2.0", etc.?
>
>   $sys_type =~ /cygwin/i
>
> Other than that, it looks good to me.
>
> -Ethan
>
>
>>
>>
>> On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:
>>
>>
>>> Hi Jeff,
>>>
>>>
>>> Sorry for the carelessness. This time it looks better? :-)
>>>
>>>
>>> Thanks,
>>> Shiqing
>>>
 Can you double check your cr / cr/lf settings?  It looks like 
you're
 committing in the windows format -- can you convert and commit 
in the

 unix format?  That way, it wouldn't look like the entire
 GNU_Install.pm file is being replaced, for example.

 I forgot to mention in my last mail -- since cmake *only* works on
 Windows and our Autotools system *doesn't* work on Windows, I 
retract
 my earlier statement: don't bother with an ompi_cmake parameter 
in the
 .ini file.  Just have OMPI.pm figure out if you're on cygwin or 
mingw

 and call the right back-end building function.

 Also, in Do_Step.pm, you might want to remove the "_" prefix 
from the

 sub names.  "_" as a prefix in perl is meant to imply that it's a
 private variable/method.

 Finally, in Do_Step.pm, do you have two _do_step() functions?


>>> Index: lib/Filesys/DiskFree.pm
>>> ===
>>> --- lib/Filesys/DiskFree.pm (revision 1271)
>>> +++ lib/Filesys/DiskFree.pm (working copy)
>>> @@ -29,6 +29,16 @@
>>> 'inodes' => "df -Pi",
>>> 'format' => "linuxish",
>>> },
>>> +'cygwin' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> +'msys' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> 'solaris' =>  {
>>> 'blocks' => "df -k",
>>> 'inodes' => "df -k -o i -F ufs",
>>> Index: lib/MTT/Common/Cmake.pm
>>> ===
>>> --- lib/MTT/Common/Cmake.pm (revision 0)
>>> +++ lib/MTT/Common/Cmake.pm (revision 0)
>>> @@ -0,0 +1,77 @@
>>> +#!/usr/bin/env perl
>>> +#
>>> +# Copyright (c) 2005-2006 The Trustees of Indiana University.
>>> +# All rights reserved.
>>> +# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
>>> +# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights 
reserved.
>>> +# Copyright (c) 2009  High Performance Computing Center 
Stuttgart,
>>> +# University of Stuttgart.  All rights 
reserved.

>>> +# $COPYRIGHT$
>>> +#
>>> +# Additional copyrights may follow
>>> +#
>>> +# $HEADER$
>>> +#
>>> +
>>> +package MTT::Common::Cmake;
>>> +my $package = ModuleName(__PACKAGE__);
>>> +
>>> +use strict;
>>> +use MTT::Messages;
>>> +use MTT::Values;
>>> +use MTT::Common::Do_step;
>>> +
>>> 
+#-- 


>>> +
>>> +# Do the following steps:
>>> +#   [ ] cmake -G "generator" -D configure_arguments source_path
>>> +#   [ ] devenv OpenMPI.sln /build
>>> +sub Install {
>>> +my ($config) = @_;
>>> +
>>> +my $x;
>>> +my $result_stdout;
>>> +my $result_stderr;
>>> +
>>> +# Prepare $ret
>>> +my $ret;
>>> +$ret->{test_result} = MTT::Values::FAIL;
>>> +$ret->{exit_status} = 0;
>>> +
>>> +# On windows, do the following steps:
>>> +
>>> +# prepare the windows style prefix.
>>> +# replace '/cygdrive/x/' with 'x:/'
>>> +my $win_prefix = substr ($config->{installdir},10,1) . ":" . 
substr

>>> ($config->{installdir},11);
>>> +
>>> +# generate Visual Studio solution files
>>> +# use 'script' to redirect MS command output
>>> +$x = MTT::Common::Do_step::do_step($config,
>>> +"script -a -c \"cmake
>>> $config->{configure_arguments} -D 
CMAKE_INSTALL_PREFIX:PATH=$win_prefix

>>> .\" -f 

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

Did you check out via https?  We only allow commits to all OMPI SVN  
repositories via https.


On Mar 12, 2009, at 5:10 AM, Shiqing Fan wrote:


Hi Ethan and Jeff,

Thanks for all your help. Now it's been fixed and tested.

But it seems that I don't have permission to commit the patch ( I got
403 forbidden ). Any idea?


Thanks,
Shiqing





Ethan Mallove wrote:
> On Wed, Mar/11/2009 03:47:22PM, Jeff Squyres wrote:
>
>> Thanks for your patience!  Yes, this looks good to me with one  
minor nit:

>>
>>
>>> +if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
>>> +$config->{compiler_name} == "microsoft") {
>>>
>> should be
>>
>>
>>> +if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
>>> +$config->{compiler_name} eq "microsoft") {
>>>
>> Josh / Ethan -- got any comments for Shiqing?
>>
>
> I just noticed last night's CYGWIN results. Nice!
>
> Should a regexp (see below) be used in case the sys_type comes  
back as, e.g.,

> "cygwin", "CYGWIN", "cygwin-2.0", etc.?
>
>   $sys_type =~ /cygwin/i
>
> Other than that, it looks good to me.
>
> -Ethan
>
>
>>
>>
>> On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:
>>
>>
>>> Hi Jeff,
>>>
>>>
>>> Sorry for the carelessness. This time it looks better? :-)
>>>
>>>
>>> Thanks,
>>> Shiqing
>>>
 Can you double check your cr / cr/lf settings?  It looks like  
you're
 committing in the windows format -- can you convert and commit  
in the

 unix format?  That way, it wouldn't look like the entire
 GNU_Install.pm file is being replaced, for example.

 I forgot to mention in my last mail -- since cmake *only* works  
on
 Windows and our Autotools system *doesn't* work on Windows, I  
retract
 my earlier statement: don't bother with an ompi_cmake parameter  
in the
 .ini file.  Just have OMPI.pm figure out if you're on cygwin or  
mingw

 and call the right back-end building function.

 Also, in Do_Step.pm, you might want to remove the "_" prefix  
from the

 sub names.  "_" as a prefix in perl is meant to imply that it's a
 private variable/method.

 Finally, in Do_Step.pm, do you have two _do_step() functions?


>>> Index: lib/Filesys/DiskFree.pm
>>>  
===

>>> --- lib/Filesys/DiskFree.pm (revision 1271)
>>> +++ lib/Filesys/DiskFree.pm (working copy)
>>> @@ -29,6 +29,16 @@
>>> 'inodes' => "df -Pi",
>>> 'format' => "linuxish",
>>> },
>>> +'cygwin' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> +'msys' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> 'solaris' =>  {
>>> 'blocks' => "df -k",
>>> 'inodes' => "df -k -o i -F ufs",
>>> Index: lib/MTT/Common/Cmake.pm
>>>  
===

>>> --- lib/MTT/Common/Cmake.pm (revision 0)
>>> +++ lib/MTT/Common/Cmake.pm (revision 0)
>>> @@ -0,0 +1,77 @@
>>> +#!/usr/bin/env perl
>>> +#
>>> +# Copyright (c) 2005-2006 The Trustees of Indiana University.
>>> +# All rights reserved.
>>> +# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights  
reserved.
>>> +# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights  
reserved.
>>> +# Copyright (c) 2009  High Performance Computing Center  
Stuttgart,
>>> +# University of Stuttgart.  All rights  
reserved.

>>> +# $COPYRIGHT$
>>> +#
>>> +# Additional copyrights may follow
>>> +#
>>> +# $HEADER$
>>> +#
>>> +
>>> +package MTT::Common::Cmake;
>>> +my $package = ModuleName(__PACKAGE__);
>>> +
>>> +use strict;
>>> +use MTT::Messages;
>>> +use MTT::Values;
>>> +use MTT::Common::Do_step;
>>> +
>>>  
+ 
#--

>>> +
>>> +# Do the following steps:
>>> +#   [ ] cmake -G "generator" -D configure_arguments source_path
>>> +#   [ ] devenv OpenMPI.sln /build
>>> +sub Install {
>>> +my ($config) = @_;
>>> +
>>> +my $x;
>>> +my $result_stdout;
>>> +my $result_stderr;
>>> +
>>> +# Prepare $ret
>>> +my $ret;
>>> +$ret->{test_result} = MTT::Values::FAIL;
>>> +$ret->{exit_status} = 0;
>>> +
>>> +# On windows, do the following steps:
>>> +
>>> +# prepare the windows style prefix.
>>> +# replace '/cygdrive/x/' with 'x:/'
>>> +my $win_prefix = substr ($config->{installdir},10,1) .  
":" . substr

>>> ($config->{installdir},11);
>>> +
>>> +# generate Visual Studio solution files
>>> +# use 'script' to redirect MS command output
>>> +$x = MTT::Common::Do_step::do_step($config,
>>> +"script -a -c \"cmake
>>> $config->{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH= 
$win_prefix

>>> .\" -f temp.txt",
>>> +$config- 
>{merge_stdout_stderr});

>>> +
>>> +# Overlapping keys in $x override 

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Hi Ethan and Jeff,

Thanks for all your help. Now it's been fixed and tested.

But it seems that I don't have permission to commit the patch ( I got 
403 forbidden ). Any idea?



Thanks,
Shiqing





Ethan Mallove wrote:

On Wed, Mar/11/2009 03:47:22PM, Jeff Squyres wrote:
  

Thanks for your patience!  Yes, this looks good to me with one minor nit:



+if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
+$config->{compiler_name} == "microsoft") {
  

should be



+if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
+$config->{compiler_name} eq "microsoft") {
  

Josh / Ethan -- got any comments for Shiqing?



I just noticed last night's CYGWIN results. Nice!

Should a regexp (see below) be used in case the sys_type comes back as, e.g.,
"cygwin", "CYGWIN", "cygwin-2.0", etc.?

  $sys_type =~ /cygwin/i

Other than that, it looks good to me.

-Ethan

  



On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:



Hi Jeff,


Sorry for the carelessness. This time it looks better? :-)


Thanks,
Shiqing
  

Can you double check your cr / cr/lf settings?  It looks like you're
committing in the windows format -- can you convert and commit in the
unix format?  That way, it wouldn't look like the entire
GNU_Install.pm file is being replaced, for example.

I forgot to mention in my last mail -- since cmake *only* works on
Windows and our Autotools system *doesn't* work on Windows, I retract
my earlier statement: don't bother with an ompi_cmake parameter in the
.ini file.  Just have OMPI.pm figure out if you're on cygwin or mingw
and call the right back-end building function.

Also, in Do_Step.pm, you might want to remove the "_" prefix from the
sub names.  "_" as a prefix in perl is meant to imply that it's a
private variable/method.

Finally, in Do_Step.pm, do you have two _do_step() functions?



Index: lib/Filesys/DiskFree.pm
===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,16 @@
'inodes' => "df -Pi",
'format' => "linuxish",
},
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
+'msys' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/Cmake.pm
===
--- lib/MTT/Common/Cmake.pm (revision 0)
+++ lib/MTT/Common/Cmake.pm (revision 0)
@@ -0,0 +1,77 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2005-2006 The Trustees of Indiana University.
+# All rights reserved.
+# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
+# Copyright (c) 2009  High Performance Computing Center Stuttgart,
+# University of Stuttgart.  All rights reserved.
+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$
+#
+
+package MTT::Common::Cmake;
+my $package = ModuleName(__PACKAGE__);
+
+use strict;
+use MTT::Messages;
+use MTT::Values;
+use MTT::Common::Do_step;
+
+#--
+
+# Do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments source_path
+#   [ ] devenv OpenMPI.sln /build
+sub Install {
+my ($config) = @_;
+
+my $x;
+my $result_stdout;
+my $result_stderr;
+
+# Prepare $ret
+my $ret;
+$ret->{test_result} = MTT::Values::FAIL;
+$ret->{exit_status} = 0;
+
+# On windows, do the following steps:
+
+# prepare the windows style prefix.
+# replace '/cygdrive/x/' with 'x:/'
+my $win_prefix = substr ($config->{installdir},10,1) . ":" . substr 
($config->{installdir},11);

+
+# generate Visual Studio solution files
+# use 'script' to redirect MS command output
+$x = MTT::Common::Do_step::do_step($config,
+"script -a -c \"cmake 
$config->{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH=$win_prefix 
.\" -f temp.txt",

+$config->{merge_stdout_stderr});
+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# compile the whole solution
+$x = MTT::Common::Do_step::do_step($config, "script -a -c 
\"devenv.com *.sln /build debug\" -f temp.txt",

+$config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# install to the prefix dir
+$x = MTT::Common::Do_step::do_step($config, "script -a -c 
\"devenv.com *.sln /project INSTALL.vcproj /build\" -f temp.txt",

+  

Re: [MTT devel] MTT on Windows

[Ethan Mallove]

On Wed, Mar/11/2009 03:47:22PM, Jeff Squyres wrote:
> Thanks for your patience!  Yes, this looks good to me with one minor nit:
>
>> +if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
>> +$config->{compiler_name} == "microsoft") {
>
> should be
>
>> +if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
>> +$config->{compiler_name} eq "microsoft") {
>
> Josh / Ethan -- got any comments for Shiqing?

I just noticed last night's CYGWIN results. Nice!

Should a regexp (see below) be used in case the sys_type comes back as, e.g.,
"cygwin", "CYGWIN", "cygwin-2.0", etc.?

  $sys_type =~ /cygwin/i

Other than that, it looks good to me.

-Ethan

>
>
>
>
> On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:
>
>> Hi Jeff,
>>
>>
>> Sorry for the carelessness. This time it looks better? :-)
>>
>>
>> Thanks,
>> Shiqing
>> >
>> > Can you double check your cr / cr/lf settings?  It looks like you're
>> > committing in the windows format -- can you convert and commit in the
>> > unix format?  That way, it wouldn't look like the entire
>> > GNU_Install.pm file is being replaced, for example.
>> >
>> > I forgot to mention in my last mail -- since cmake *only* works on
>> > Windows and our Autotools system *doesn't* work on Windows, I retract
>> > my earlier statement: don't bother with an ompi_cmake parameter in the
>> > .ini file.  Just have OMPI.pm figure out if you're on cygwin or mingw
>> > and call the right back-end building function.
>> >
>> > Also, in Do_Step.pm, you might want to remove the "_" prefix from the
>> > sub names.  "_" as a prefix in perl is meant to imply that it's a
>> > private variable/method.
>> >
>> > Finally, in Do_Step.pm, do you have two _do_step() functions?
>> >
>>
>> Index: lib/Filesys/DiskFree.pm
>> ===
>> --- lib/Filesys/DiskFree.pm  (revision 1271)
>> +++ lib/Filesys/DiskFree.pm  (working copy)
>> @@ -29,6 +29,16 @@
>>  'inodes' => "df -Pi",
>>  'format' => "linuxish",
>> },
>> +'cygwin' => {
>> +'blocks' => "df -P",
>> +'inodes' => "df -Pi",
>> +'format' => "linuxish",
>> +},
>> +'msys' => {
>> +'blocks' => "df -P",
>> +'inodes' => "df -Pi",
>> +'format' => "linuxish",
>> +},
>> 'solaris' =>  {
>>  'blocks' => "df -k",
>>  'inodes' => "df -k -o i -F ufs",
>> Index: lib/MTT/Common/Cmake.pm
>> ===
>> --- lib/MTT/Common/Cmake.pm  (revision 0)
>> +++ lib/MTT/Common/Cmake.pm  (revision 0)
>> @@ -0,0 +1,77 @@
>> +#!/usr/bin/env perl
>> +#
>> +# Copyright (c) 2005-2006 The Trustees of Indiana University.
>> +# All rights reserved.
>> +# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
>> +# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
>> +# Copyright (c) 2009  High Performance Computing Center Stuttgart,
>> +# University of Stuttgart.  All rights reserved.
>> +# $COPYRIGHT$
>> +#
>> +# Additional copyrights may follow
>> +#
>> +# $HEADER$
>> +#
>> +
>> +package MTT::Common::Cmake;
>> +my $package = ModuleName(__PACKAGE__);
>> +
>> +use strict;
>> +use MTT::Messages;
>> +use MTT::Values;
>> +use MTT::Common::Do_step;
>> +
>> +#--
>> +
>> +# Do the following steps:
>> +#   [ ] cmake -G "generator" -D configure_arguments source_path
>> +#   [ ] devenv OpenMPI.sln /build
>> +sub Install {
>> +my ($config) = @_;
>> +
>> +my $x;
>> +my $result_stdout;
>> +my $result_stderr;
>> +
>> +# Prepare $ret
>> +my $ret;
>> +$ret->{test_result} = MTT::Values::FAIL;
>> +$ret->{exit_status} = 0;
>> +
>> +# On windows, do the following steps:
>> +
>> +# prepare the windows style prefix.
>> +# replace '/cygdrive/x/' with 'x:/'
>> +my $win_prefix = substr ($config->{installdir},10,1) . ":" . substr 
>> ($config->{installdir},11);
>> +
>> +# generate Visual Studio solution files
>> +# use 'script' to redirect MS command output
>> +$x = MTT::Common::Do_step::do_step($config,
>> +"script -a -c \"cmake 
>> $config->{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH=$win_prefix 
>> .\" -f temp.txt",
>> +$config->{merge_stdout_stderr});
>> +
>> +# Overlapping keys in $x override $ret
>> +%$ret = (%$ret, %$x);
>> +return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
>> +
>> +# compile the whole solution
>> +$x = MTT::Common::Do_step::do_step($config, "script -a -c 
>> \"devenv.com *.sln /build debug\" -f temp.txt",
>> +$config->{merge_stdout_stderr});
>> +%$ret = (%$ret, %$x);
>> +return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
>> +
>> +# install to the prefix dir
>> +$x = MTT::Common::Do_step::do_step($config, 

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

Thanks for your patience!  Yes, this looks good to me with one minor  
nit:



+if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
+$config->{compiler_name} == "microsoft") {


should be


+if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
+$config->{compiler_name} eq "microsoft") {


Josh / Ethan -- got any comments for Shiqing?




On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:


Hi Jeff,


Sorry for the carelessness. This time it looks better? :-)


Thanks,
Shiqing
>
> Can you double check your cr / cr/lf settings?  It looks like you're
> committing in the windows format -- can you convert and commit in  
the

> unix format?  That way, it wouldn't look like the entire
> GNU_Install.pm file is being replaced, for example.
>
> I forgot to mention in my last mail -- since cmake *only* works on
> Windows and our Autotools system *doesn't* work on Windows, I  
retract
> my earlier statement: don't bother with an ompi_cmake parameter in  
the
> .ini file.  Just have OMPI.pm figure out if you're on cygwin or  
mingw

> and call the right back-end building function.
>
> Also, in Do_Step.pm, you might want to remove the "_" prefix from  
the

> sub names.  "_" as a prefix in perl is meant to imply that it's a
> private variable/method.
>
> Finally, in Do_Step.pm, do you have two _do_step() functions?
>

Index: lib/Filesys/DiskFree.pm
===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,16 @@
'inodes' => "df -Pi",
'format' => "linuxish",
},
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
+'msys' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/Cmake.pm
===
--- lib/MTT/Common/Cmake.pm (revision 0)
+++ lib/MTT/Common/Cmake.pm (revision 0)
@@ -0,0 +1,77 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2005-2006 The Trustees of Indiana University.
+# All rights reserved.
+# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights  
reserved.
+# Copyright (c) 2009  High Performance Computing Center  
Stuttgart,
+# University of Stuttgart.  All rights  
reserved.

+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$
+#
+
+package MTT::Common::Cmake;
+my $package = ModuleName(__PACKAGE__);
+
+use strict;
+use MTT::Messages;
+use MTT::Values;
+use MTT::Common::Do_step;
+
+ 
#--

+
+# Do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments source_path
+#   [ ] devenv OpenMPI.sln /build
+sub Install {
+my ($config) = @_;
+
+my $x;
+my $result_stdout;
+my $result_stderr;
+
+# Prepare $ret
+my $ret;
+$ret->{test_result} = MTT::Values::FAIL;
+$ret->{exit_status} = 0;
+
+# On windows, do the following steps:
+
+# prepare the windows style prefix.
+# replace '/cygdrive/x/' with 'x:/'
+my $win_prefix = substr ($config->{installdir},10,1) . ":" .  
substr ($config->{installdir},11);

+
+# generate Visual Studio solution files
+# use 'script' to redirect MS command output
+$x = MTT::Common::Do_step::do_step($config,
+"script -a -c \"cmake  
$config->{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH= 
$win_prefix .\" -f temp.txt",
+$config- 
>{merge_stdout_stderr});

+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# compile the whole solution
+$x = MTT::Common::Do_step::do_step($config, "script -a -c  
\"devenv.com *.sln /build debug\" -f temp.txt",
+$config- 
>{merge_stdout_stderr});

+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# install to the prefix dir
+$x = MTT::Common::Do_step::do_step($config, "script -a -c  
\"devenv.com *.sln /project INSTALL.vcproj /build\" -f temp.txt",
+$config- 
>{merge_stdout_stderr});

+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+}
+
+1;
Index: lib/MTT/Common/Do_step.pm
===
--- lib/MTT/Common/Do_step.pm   (revision 0)
+++ 

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

On Mar 11, 2009, at 7:32 AM, Shiqing Fan wrote:


That's an old patch, Here attatched is the new one.



Ah, excellent.


> Can we think of a better test here?  Will we always be running MTT
> under Cygwin?

On windows, there isn't any choice to run MTT in a linux-shell,  
another
option is MinGW, but I didn't try it. Personally, I think Cygwin is  
the
best choice, because it provides full Linux-like environment. Note  
that,
we use quite a lot Linux shell command in MTT, like df, uname, and  
so on.


So we can test like:
if (`uname -o` == "Cygwin" ||
`uname -o` == "MinGW" &&
$config->{compiler_name} == "microsoft") {
.



Good point.  I see that your new patch doesn't test for MinGW --  
should it?



Index: lib/Filesys/DiskFree.pm

===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,11 @@
'inodes' => "df -Pi",
'format' => "linuxish",
},
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",


Do you need MinGW in this patch? ^^^



Index: lib/MTT/Common/GNU_Install.pm
===
--- lib/MTT/Common/GNU_Install.pm   (revision 1271)
+++ lib/MTT/Common/GNU_Install.pm   (working copy)
@@ -44,6 +44,51 @@
$ret->{test_result} = MTT::Values::FAIL;
$ret->{exit_status} = 0;

+if (`uname -o` == "Cygwin" && $config->{compiler_name} ==  
"microsoft") {


Per above, do we need MinGW in this test? ^^



+# On windows, do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments  
source_path

+#   [ ] devenv OpenMPI.sln /build
+
+# prepare the windows style prefix.
+# replace '/cygdrive/x/' with 'x:/'
+my $win_prefix = substr ($config->{installdir},10,1) .  
":" . substr ($config->{installdir},11);

+
+# generate Visual Studio solution files
+# use 'script' to redirect MS command output
+$x = _do_step($config,
+  "script -a -c \"cmake $config- 
>{configure_arguments} -D CMAKE_INSTALL_PREFIX:PATH=$win_prefix .\" - 
f temp.txt",

+  $config->{merge_stdout_stderr});
+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+if (exists($ENV{LD_LIBRARY_PATH})) {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}: 
$ENV{LD_LIBRARY_PATH}";

+} else {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
+}



+
+# compile the whole solution
+$x = _do_step($config, "script -a -c \"devenv.com *.sln / 
build debug\" -f temp.txt",

+  $config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));


I missed this the first time -- don't you need to restore %ENV here?


+
+# install to the prefix dir
+$x = _do_step($config, "script -a -c \"devenv.com *.sln / 
project INSTALL.vcproj /build\" -f temp.txt",

+  $config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+}
+


I think I would be more comfortable separating this stuff into a  
Cmake.pm or somesuch.  After all, it's *not* a configure/make (i.e.,  
GNU tools) build.  I think the default should call the GNU install  
process, but if you set ompi_cmake = 1 (or somesuch), then call your  
module/routine instead of the GNU one.  For example:


my $install;
if ($config->{ompi_cmake}) {
$install = MTT::Common::Cmake::Install($gnu);
} else {
$install = MTT::Common::GNU_Install::Install($gnu);
}

Just curious -- does the OMPI cmake system also work on Linux?  I ask  
because your cmake process is fairly win-specific.  If it's *only*  
intended to work on windoze, perhaps we should add some if tests to it  
that warn/error if you're *not* running on cygwin/mingw (this is only  
relevant if/when the cmake stuff moves to its own .pm file).


Make sense?


# If the user does not use --prefix on their own, default
# to $installdir
my $prefix;
Index: lib/MTT/Defaults.pm
===
--- lib/MTT/Defaults.pm (revision 1271)
+++ lib/MTT/Defaults.pm (working copy)
@@ -39,7 +39,7 @@
},

known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai",  
"absoft",
-  "pathscale", "sun", 

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Should this code rather be in a different .pm and the user selects 
which one to use via the .ini file?


Yes, that's also what I thought. Then we need to change a little in 
upper level, e.g. OMPI.pm, we should test the environment and 
compiler, in order to choose the right install module (Gnu_install.pm 
or Win_install.pm maybe).


Sorry, I might be wrong on this point, actually I tried, but I didn't 
figure out how to do that.


Thanks,
Shiqing

Re: [MTT devel] MTT on Windows

[Shiqing Fan]

Hi Jeff,

That's an old patch, Here attatched is the new one.

I was just writing the email to be sent to MTT-devel, now I think it's 
better to put it in this thread.



===
--- lib/MTT/Common/GNU_Install.pm(revision 1271)
+++ lib/MTT/Common/GNU_Install.pm(working copy)
@@ -43,7 +43,45 @@
my $ret;
$ret->{test_result} = MTT::Values::FAIL;
$ret->{exit_status} = 0;
+
+if (`uname -o` == "Cygwin") {


Can we think of a better test here?  Will we always be running MTT 
under Cygwin?


On windows, there isn't any choice to run MTT in a linux-shell, another 
option is MinGW, but I didn't try it. Personally, I think Cygwin is the 
best choice, because it provides full Linux-like environment. Note that, 
we use quite a lot Linux shell command in MTT, like df, uname, and so on.


So we can test like:
   if (`uname -o` == "Cygwin" ||
   `uname -o` == "MinGW" &&
   $config->{compiler_name} == "microsoft") {
   .

Here in the new patch, I introduce the variable compiler_name, in order 
to make it possible to use other compilers under Cygwin, e.g. gcc.


Should this code rather be in a different .pm and the user selects 
which one to use via the .ini file?


Yes, that's also what I thought. Then we need to change a little in 
upper level, e.g. OMPI.pm, we should test the environment and compiler, 
in order to choose the right install module (Gnu_install.pm or 
Win_install.pm maybe).




+if (exists($ENV{LD_LIBRARY_PATH})) {
+$ENV{LD_LIBRARY_PATH} = 
"$config->{libdir}:$ENV{LD_LIBRARY_PATH}";

+} else {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
+}
+
+# compile the whole solution
+$x = _do_step($config, "devenv.com OpenMPI.sln /build 
debug", 1);

+%$ret = (%$ret, %$x);
+return $ret if 
(!MTT::DoCommand::wsuccess($ret->{exit_status}));

+
+# install, not working yet


What does this comment mean? ^^^


It remained unfixed in that patch. See the new one :-).


Thanks,
Shiqing





Index: lib/Filesys/DiskFree.pm
===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,11 @@
'inodes' => "df -Pi",
'format' => "linuxish",
 },
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
 'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/GNU_Install.pm
===
--- lib/MTT/Common/GNU_Install.pm   (revision 1271)
+++ lib/MTT/Common/GNU_Install.pm   (working copy)
@@ -44,6 +44,51 @@
 $ret->{test_result} = MTT::Values::FAIL;
 $ret->{exit_status} = 0;
 
+if (`uname -o` == "Cygwin" && $config->{compiler_name} == "microsoft") {
+# On windows, do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments source_path
+#   [ ] devenv OpenMPI.sln /build
+
+# prepare the windows style prefix.
+# replace '/cygdrive/x/' with 'x:/'
+my $win_prefix = substr ($config->{installdir},10,1) . ":" . substr 
($config->{installdir},11);
+
+# generate Visual Studio solution files
+# use 'script' to redirect MS command output
+$x = _do_step($config,
+  "script -a -c \"cmake $config->{configure_arguments} -D 
CMAKE_INSTALL_PREFIX:PATH=$win_prefix .\" -f temp.txt", 
+  $config->{merge_stdout_stderr});
+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+if (exists($ENV{LD_LIBRARY_PATH})) {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}:$ENV{LD_LIBRARY_PATH}";
+} else {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
+}
+
+# compile the whole solution
+$x = _do_step($config, "script -a -c \"devenv.com *.sln /build debug\" 
-f temp.txt",
+  $config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# install to the prefix dir
+$x = _do_step($config, "script -a -c \"devenv.com *.sln /project 
INSTALL.vcproj /build\" -f temp.txt",
+  $config->{merge_stdout_stderr});
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret->{exit_status}));
+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+}
+
 # If the user does not use --prefix on their own, default
 # to $installdir
 my $prefix;
Index: lib/MTT/Defaults.pm

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

(moving to mtt-devel)

Shiqing --

In looking at your patch, I see a few nits.  See below.


On Feb 26, 2009, at 12:56 PM, Shiqing Fan wrote:


Index: lib/Filesys/DiskFree.pm

===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,11 @@
'inodes' => "df -Pi",
'format' => "linuxish",
},
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/GNU_Install.pm
===
--- lib/MTT/Common/GNU_Install.pm   (revision 1271)
+++ lib/MTT/Common/GNU_Install.pm   (working copy)
@@ -43,7 +43,45 @@
my $ret;
$ret->{test_result} = MTT::Values::FAIL;
$ret->{exit_status} = 0;
+
+if (`uname -o` == "Cygwin") {


Can we think of a better test here?  Will we always be running MTT  
under Cygwin?


Should this code rather be in a different .pm and the user selects  
which one to use via the .ini file?




+# On windows, do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments  
source_path

+#   [ ] devenv OpenMPI.sln /build

+# generate Visual Studio solution files
+$x = _do_step($config,
+  "cmake $config->{configure_arguments} -D  
CMAKE_INSTALL_PREFIX:PATH=$config->{installdir} .", 1);

+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+if (exists($ENV{LD_LIBRARY_PATH})) {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}: 
$ENV{LD_LIBRARY_PATH}";

+} else {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
+}
+
+# compile the whole solution
+$x = _do_step($config, "devenv.com OpenMPI.sln /build  
debug", 1);

+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+# install, not working yet


What does this comment mean? ^^^



+$x = _do_step($config, "devenv INSTALL.vcproj /build", 1);
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+
+}
+
# If the user does not use --prefix on their own, default
# to $installdir
my $prefix;
@@ -224,5 +262,3 @@
my ($cmd, $step) = @_;
return MTT::DoCommand::RunStep(1, $cmd, 30, undef, undef, $step);
}
-
-1;


Don't remove the "1;" at the end of the file...



Index: lib/MTT/Defaults.pm
===
--- lib/MTT/Defaults.pm (revision 1271)
+++ lib/MTT/Defaults.pm (working copy)
@@ -39,7 +39,7 @@
},

known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai",  
"absoft",
-  "pathscale", "sun", "none",  
"unknown" ],
+  "pathscale", "sun", "microsoft",  
"none", "unknown" ],
known_resource_manager_names => [ "slurm", "tm", "loadleveler",  
"n1ge",

  "alps", "none", "unknown" ],
known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx",  
"verbs",



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT bugs] [MTT] #258: Compare non-contiguous date ranges

[Jeff Squyres]

Hey, how about Javascript calendars for selecting date ranges?

:p


On Nov 11, 2008, at 1:15 PM, Ethan Mallove wrote:


Folks,

I have a fix for this ticket. The syntax to get
non-contiguous date ranges is to use space-delimited quoted
tokens (see http://tinyurl.com/6e2pqa), e.g.,

 "2007-10-29 - 2007-11-30" "2008-11-07 - 2008-11-14" "by month"

Trying to view year-old results alongside week-old results in the
current Reporter can result in 10+ minutes of database thrashing. With
this fix, it's around 30 seconds. If I don't get any bug reports about
the above ~emallove Reporter within 48 hours, I'll push it out to the
main www.open-mpi.org site.

Thanks,
Ethan


On Wed, Aug/08/2007 07:22:14PM, MTT wrote:

#258: Compare non-contiguous date ranges
- 
+--

Reporter:  jjhursey |   Owner:
Type:  defect   |  Status:  new
Priority:  major|   Milestone:  Future
Component:  Server side  | Version:  trunk
Keywords:   |
- 
+--
It is possible to get results from two types of runs (say threaded  
and
non-threaded) by regular expression arguments to some existing  
reporter

fields (i,e., configure arguments).

However it is currently not possible to compare two distinct date  
ranges.

We should extend the reporter to effectively display such results.

We could extend the date specification to allow for a comma  
separated list

of ranges:
{{{
To compare results from July 20, 21, and 22:
  2007-07-20 - 2007-07-21,2007-07-21 - 2007-07-22,2007-07-22 -  
2007-07-23

}}}

Then the reporter could make distinct summary tables for each  
range. Or

something like that.

--
Ticket URL: 
MTT 
Issue tracking for the MPI Testing Tool.




--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Mike Dubman]

yep. works as a charm

On Tue, Oct 28, 2008 at 4:57 PM, Jeff Squyres  wrote:

> I sent the whatami patch upstream, and Brian Finley (the whatami author)
> encouraged us to actually use his recently-included Centos-5 support instead
> of our patch.  This is because his support is generic enough that it should
> work for any lsb_release-capable machine (to include Centos 5).
>
> I pulled that down into the MTT trunk; Mike, could you verify that it works
> for you?
>
>
>
> On Oct 28, 2008, at 8:30 AM, Jeff Squyres (jsquyres) wrote:
>
>  Done!
>>
>> On Oct 28, 2008, at 2:06 AM, Mike Dubman wrote:
>>
>> >
>> > Hey Jeff,
>> >
>> > I have no svn permissions to commit. Can you please provide me with
>> > one? (login: miked)
>> > Thanks
>> >
>> > On Mon, Oct 27, 2008 at 4:38 PM, Jeff Squyres 
>> > wrote:
>> > Aside from the 2 space tabs, looks great.  ;-)
>> >
>> > Go ahead and commit; I'll send the whatami patch upstream (whatami
>> > is maintained by Brian Finley at Argonne National Labs).
>> >
>> >
>> >
>> > On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:
>> >
>> >
>> > Hello guys,
>> >
>> > Please consider applying attached mtt patch to allow following
>> > features:
>> >
>> >• Support for centos5
>> >• Send single, digested email report for all completed tests
>> > (similar to text file summary file)
>> >• Provide basic statistics in the digested email about
>> > completed tests (similar to junit): duration, mpi version, overall
>> > status.
>> >
>> > Thanks
>> >
>> > Mike
>> > 
>> >
>> > ___
>> > mtt-devel mailing list
>> > mtt-de...@open-mpi.org
>> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>> >
>> >
>> > --
>> > Jeff Squyres
>> > Cisco Systems
>> >
>> >
>> >
>> > ___
>> > mtt-devel mailing list
>> > mtt-de...@open-mpi.org
>> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>> >
>> > ___
>> > mtt-devel mailing list
>> > mtt-de...@open-mpi.org
>> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>>
>> --
>> Jeff Squyres
>> Cisco Systems
>>
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>
>
> --
> Jeff Squyres
> Cisco Systems
>
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

I sent the whatami patch upstream, and Brian Finley (the whatami  
author) encouraged us to actually use his recently-included Centos-5  
support instead of our patch.  This is because his support is generic  
enough that it should work for any lsb_release-capable machine (to  
include Centos 5).


I pulled that down into the MTT trunk; Mike, could you verify that it  
works for you?



On Oct 28, 2008, at 8:30 AM, Jeff Squyres (jsquyres) wrote:


Done!

On Oct 28, 2008, at 2:06 AM, Mike Dubman wrote:

>
> Hey Jeff,
>
> I have no svn permissions to commit. Can you please provide me with
> one? (login: miked)
> Thanks
>
> On Mon, Oct 27, 2008 at 4:38 PM, Jeff Squyres 
> wrote:
> Aside from the 2 space tabs, looks great.  ;-)
>
> Go ahead and commit; I'll send the whatami patch upstream (whatami
> is maintained by Brian Finley at Argonne National Labs).
>
>
>
> On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:
>
>
> Hello guys,
>
> Please consider applying attached mtt patch to allow following
> features:
>
>• Support for centos5
>• Send single, digested email report for all completed tests
> (similar to text file summary file)
>• Provide basic statistics in the digested email about
> completed tests (similar to junit): duration, mpi version, overall
> status.
>
> Thanks
>
> Mike
> 
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
>
> --
> Jeff Squyres
> Cisco Systems
>
>
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

Done!

On Oct 28, 2008, at 2:06 AM, Mike Dubman wrote:



Hey Jeff,

I have no svn permissions to commit. Can you please provide me with  
one? (login: miked)

Thanks

On Mon, Oct 27, 2008 at 4:38 PM, Jeff Squyres   
wrote:

Aside from the 2 space tabs, looks great.  ;-)

Go ahead and commit; I'll send the whatami patch upstream (whatami  
is maintained by Brian Finley at Argonne National Labs).




On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:


Hello guys,

Please consider applying attached mtt patch to allow following  
features:


   • Support for centos5
   • Send single, digested email report for all completed tests  
(similar to text file summary file)
   • Provide basic statistics in the digested email about  
completed tests (similar to junit): duration, mpi version, overall  
status.


Thanks

Mike


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems



___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

Aside from the 2 space tabs, looks great.  ;-)

Go ahead and commit; I'll send the whatami patch upstream (whatami is  
maintained by Brian Finley at Argonne National Labs).



On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:



Hello guys,

Please consider applying attached mtt patch to allow following  
features:


• Support for centos5
	• Send single, digested email report for all completed tests  
(similar to text file summary file)
	• Provide basic statistics in the digested email about completed  
tests (similar to junit): duration, mpi version, overall status.


Thanks

Mike

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1216

[Ethan Mallove]

Nice!

Which device(s) does DiskFree.pm check for space on? I
occasionally run out of room in /tmp, so I would want it to
check my swap space.

Can we remove these? (All but the .pm and README files.)

 trunk/lib/Filesys/Changes
 trunk/lib/Filesys/MANIFEST
 trunk/lib/Filesys/Makefile.PL
 trunk/lib/Filesys/eg/
 trunk/lib/Filesys/eg/perldf
 trunk/lib/Filesys/eg/silly
 trunk/lib/Filesys/test.pl

-Ethan

On Tue, Jul/22/2008 09:27:04PM, jsquy...@osl.iu.edu wrote:
> Author: jsquyres
> Date: 2008-07-22 21:27:04 EDT (Tue, 22 Jul 2008)
> New Revision: 1216
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1216
> 
> Log:
> Fixes #370: have MTT exit if there isn't enough disk space left.
> 
> Added:
>trunk/lib/Filesys/
>trunk/lib/Filesys/Changes
>trunk/lib/Filesys/DiskFree.pm
>trunk/lib/Filesys/MANIFEST
>trunk/lib/Filesys/Makefile.PL
>trunk/lib/Filesys/README
>trunk/lib/Filesys/README.mtt
>trunk/lib/Filesys/eg/
>trunk/lib/Filesys/eg/perldf   (contents, props changed)
>trunk/lib/Filesys/eg/silly   (contents, props changed)
>trunk/lib/Filesys/test.pl
> Text files modified: 
>trunk/client/mtt |11   
>   
>trunk/lib/MTT/Globals.pm | 5 +++   
>   
>trunk/lib/MTT/MPI/Get.pm | 2   
>   
>trunk/lib/MTT/MPI/Install.pm | 4 +-
>   
>trunk/lib/MTT/Test/Build.pm  | 4 +-
>   
>trunk/lib/MTT/Test/Get.pm| 2   
>   
>trunk/lib/MTT/Test/Run.pm| 4 +-
>   
>trunk/lib/MTT/Test/RunEngine.pm  | 6 ++--  
>   
>trunk/lib/MTT/Util.pm|50 
> ++- 
>trunk/samples/ompi-core-template.ini |17 + 
>   
>10 files changed, 86 insertions(+), 19 deletions(-)
> 
> Modified: trunk/client/mtt
> ==
> --- trunk/client/mtt  (original)
> +++ trunk/client/mtt  2008-07-22 21:27:04 EDT (Tue, 22 Jul 2008)
> @@ -112,6 +112,7 @@
>  use MTT::Values;
>  use MTT::Timer;
>  use MTT::Util;
> +use Filesys::DiskFree;
>  
>  my @file_arg;
>  my $stdin_arg;
> @@ -492,31 +493,31 @@
>  MTT::Reporter::Init($ini);
>  }
>  
> -if ($mpi_get && !MTT::Util::find_terminate_file()) {
> +if ($mpi_get && !MTT::Util::time_to_terminate()) {
>  ::Timer::start($time_phases);
>  MTT::MPI::Get::Get($ini, $source_dir, $force_arg);
>  ::Timer::stop();
>  ::Timer::print("Phase: MPI Get", $time_phases, 1);
>  }
> -if ($mpi_install && !MTT::Util::find_terminate_file()) {
> +if ($mpi_install && !MTT::Util::time_to_terminate()) {
>  ::Timer::start($time_phases);
>  MTT::MPI::Install::Install($ini, $ini_full, $install_dir, 
> $force_arg);
>  ::Timer::stop();
>  ::Timer::print("Phase: MPI Install", $time_phases, 1);
>  }
> -if ($test_get && !MTT::Util::find_terminate_file()) {
> +if ($test_get && !MTT::Util::time_to_terminate()) {
>  ::Timer::start($time_phases);
>  MTT::Test::Get::Get($ini, $source_dir, $force_arg);
>  ::Timer::stop();
>  ::Timer::print("Phase: Test Get", $time_phases, 1);
>  }
> -if ($test_build && !MTT::Util::find_terminate_file()) {
> +if ($test_build && !MTT::Util::time_to_terminate()) {
>  ::Timer::start($time_phases);
>  MTT::Test::Build::Build($ini, $ini_full, $install_dir, $force_arg);
>  ::Timer::stop();
>  ::Timer::print("Phase: Test Build", $time_phases, 1);
>  }
> -if ($test_run && !MTT::Util::find_terminate_file()) {
> +if ($test_run && !MTT::Util::time_to_terminate()) {
>  ::Timer::start($time_phases);
>  MTT::Test::Run::Run($ini, $ini_full, $install_dir, $runs_data_dir,
>  $force_arg);
> 
> Added: trunk/lib/Filesys/Changes
> ==
> --- (empty file)
> +++ trunk/lib/Filesys/Changes 2008-07-22 21:27:04 EDT (Tue, 22 Jul 2008)
> @@ -0,0 +1,27 @@
> +Revision history for Perl extension Filesys::DiskFree.
> +
> +0.06   Fri Oct 23 18:26:19 EDT 1998
> + - Now supports HP-UX, thanks to Richard Paul for the patch.
> +
> +0.05   Wed Aug 26 23:54:15 EDT 1998
> + - Now supports OSF/1, thanks to Eric Foster-Johnson for the details.
> +
> +0.04   Sat Aug  1 12:15:42 EDT 1998
> + - Fixes problem if passed a barefilename, without any '/'.
> + - Bug fix in 'silly' example, accidentally deleted a ;
> + - perldf now works on filesystems with 0% free (eg psuedo-fs), and
> +   formats filesystems with <10% free 

Re: [MTT devel] [MTT bugs] [MTT] #355: tooltips for reporter

[Jeff Squyres]

Ethan and I played with this a bit more and it might not be workable.   
See #355 for more details.


On Apr 21, 2008, at 2:06 PM, Ethan Mallove wrote:


Do these work for you?

 http://tinyurl.com/49m2n4

They work for me in IE (Windows) and Mozilla (Solaris), but
not in Firefox and Opera. The joys of JavaScript :-)

-Ethan


On Mon, Apr/21/2008 01:32:49PM, MTT wrote:

#355: tooltips for reporter
- 
+--

Reporter:  jsquyres |   Owner:
Type:  enhancement  |  Status:  new
Priority:  major|   Milestone:  v3.1
Component:  Server side  | Version:  trunk
Keywords:   |
- 
+--
Currently, when browsing the reporter, if you hover over a link in  
the

reporter, the browser (or mail client) may show the entire URL of the
hyperlink as a tooltip (see attached screenshot).  Is there a way  
to show
something nicer / smaller?  In the attached screenshot, either no  
tooltip
or a tooltip showing "Drill down to all 1.2.7a results" would  
be nice.


--
Ticket URL: 
MTT 
Issue tracking for the MPI Testing Tool.


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT bugs] [MTT] #355: tooltips for reporter

[Ethan Mallove]

Do these work for you?

  http://tinyurl.com/49m2n4

They work for me in IE (Windows) and Mozilla (Solaris), but
not in Firefox and Opera. The joys of JavaScript :-)

-Ethan


On Mon, Apr/21/2008 01:32:49PM, MTT wrote:
> #355: tooltips for reporter
> -+--
>  Reporter:  jsquyres |   Owner:   
>  Type:  enhancement  |  Status:  new  
>  Priority:  major|   Milestone:  v3.1 
> Component:  Server side  | Version:  trunk
>  Keywords:   |  
> -+--
>  Currently, when browsing the reporter, if you hover over a link in the
>  reporter, the browser (or mail client) may show the entire URL of the
>  hyperlink as a tooltip (see attached screenshot).  Is there a way to show
>  something nicer / smaller?  In the attached screenshot, either no tooltip
>  or a tooltip showing "Drill down to all 1.2.7a results" would be nice.
> 
> -- 
> Ticket URL: 
> MTT 
> Issue tracking for the MPI Testing Tool.
> 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1176

[Jeff Squyres]

Um -- yeah, probably.  :-)

But there's also likely no harm in leaving them there.  :-)


On Apr 4, 2008, at 4:29 PM, Ethan Mallove wrote:

I like the "all" keyword. Are these no longer needed?

 _mpi_get_names()
 _mpi_install_names()
 _test_get_names()
 _test_build_names()

-Ethan

On Fri, Apr/04/2008 03:31:07PM, jsquy...@osl.iu.edu wrote:

Author: jsquyres
Date: 2008-04-04 15:31:07 EDT (Fri, 04 Apr 2008)
New Revision: 1176
URL: https://svn.open-mpi.org/trac/mtt/changeset/1176

Log:
Allow "all" keyword in mpi_get, test_get, and test_build fields.

Text files modified:
  trunk/CHANGES| 5 +
  trunk/lib/MTT/MPI/Install.pm | 9 ++---
  trunk/lib/MTT/Test/Build.pm  | 9 ++---
  trunk/lib/MTT/Test/Run.pm| 6 --
  4 files changed, 21 insertions(+), 8 deletions(-)

Modified: trunk/CHANGES
=
=
=
=
=
=
=
=
=
=
--- trunk/CHANGES   (original)
+++ trunk/CHANGES   2008-04-04 15:31:07 EDT (Fri, 04 Apr 2008)
@@ -50,6 +50,11 @@
- _details() - pass arbitrary values from test run sections
   to the mpi details section, indexed by string

+- Allow mpi_get_name, test_get_name, and test_build_name fields to
+  accept the special value "all", meaning that they'll use all
+  corresponding sections that are found (vs. needing to list every
+  section explicitly)
+
- Added export for MTT_TEST_EXECUTABLE, may be used for clean up  
after

  mpi process : pkill -9 $MTT_TEST_EXECUTABLE


Modified: trunk/lib/MTT/MPI/Install.pm
= 
= 
= 
= 
= 
= 
= 
= 
= 
=

--- trunk/lib/MTT/MPI/Install.pm(original)
+++ trunk/lib/MTT/MPI/Install.pm	2008-04-04 15:31:07 EDT (Fri, 04  
Apr 2008)

@@ -199,7 +199,8 @@

# This is only warning about the INI file; we'll see
# if we find meta data for the MPI get later
-if (!$ini_full->SectionExists("mpi get:  
$mpi_get_name")) {

+if ($mpi_get_name ne "all" &&
+!$ini_full->SectionExists("mpi get:  
$mpi_get_name")) {
Warning("Warning: MPI Get section  
\"$mpi_get_name\" does not seem to exist in the INI file\n");

}

@@ -207,7 +208,8 @@
# skip it.  Don't issue a warning because command  
line

# parameters may well have dictated to skip this MPI
# get section.
-if (!exists($MTT::MPI::sources->{$mpi_get_name})) {
+if ($mpi_get_name ne "all" &&
+!exists($MTT::MPI::sources->{$mpi_get_name})) {
Debug("Have no sources for MPI Get  
\"$mpi_get_name\", skipping\n");

next;
}
@@ -217,7 +219,8 @@

# For each MPI source
foreach my $mpi_get_key (keys(% 
{$MTT::MPI::sources})) {

-if ($mpi_get_key eq $mpi_get_name) {
+if ($mpi_get_name eq "all" ||
+$mpi_get_key eq $mpi_get_name) {

# For each version of that source
my $mpi_get = $MTT::MPI::sources- 
>{$mpi_get_key};


Modified: trunk/lib/MTT/Test/Build.pm
= 
= 
= 
= 
= 
= 
= 
= 
= 
=

--- trunk/lib/MTT/Test/Build.pm (original)
+++ trunk/lib/MTT/Test/Build.pm	2008-04-04 15:31:07 EDT (Fri, 04  
Apr 2008)

@@ -120,7 +120,8 @@

# This is only warning about the INI file; we'll see
# if we find meta data for the test get later
-if (!$ini_full->SectionExists("test get:  
$test_get_name")) {

+if ($test_get_name ne "all" &&
+!$ini_full->SectionExists("test get:  
$test_get_name")) {
Warning("Test Get section \"$test_get_name\"  
does not seem to exist in the INI file\n");

}

@@ -128,14 +129,16 @@
# skip it.  Don't issue a warning because command  
line

# parameters may well have dictated to skip this
# section.
-if (!exists($MTT::Test::sources- 
>{$test_get_name})) {

+if ($test_get_name ne "all" &&
+!exists($MTT::Test::sources- 
>{$test_get_name})) {
Debug("Have no sources for Test Get  
\"$test_get_name\", skipping\n");

next;
}

# Find the matching test source
foreach my $test_get_key (keys(% 
{$MTT::Test::sources})) {

-if ($test_get_key eq $test_get_name) {
+if ($test_get_name eq "all" ||
+$test_get_key eq $test_get_name) {
my $test_get = $MTT::Test::sources- 
>{$test_get_key};


# For each MPI source

Modified: trunk/lib/MTT/Test/Run.pm
= 
= 
= 
= 
= 
= 
= 
= 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1176

[Ethan Mallove]

I like the "all" keyword. Are these no longer needed?

  _mpi_get_names()
  _mpi_install_names()
  _test_get_names()
  _test_build_names()

-Ethan

On Fri, Apr/04/2008 03:31:07PM, jsquy...@osl.iu.edu wrote:
> Author: jsquyres
> Date: 2008-04-04 15:31:07 EDT (Fri, 04 Apr 2008)
> New Revision: 1176
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1176
> 
> Log:
> Allow "all" keyword in mpi_get, test_get, and test_build fields.
> 
> Text files modified: 
>trunk/CHANGES| 5 + 
>   
>trunk/lib/MTT/MPI/Install.pm | 9 ++--- 
>   
>trunk/lib/MTT/Test/Build.pm  | 9 ++--- 
>   
>trunk/lib/MTT/Test/Run.pm| 6 --
>   
>4 files changed, 21 insertions(+), 8 deletions(-)
> 
> Modified: trunk/CHANGES
> ==
> --- trunk/CHANGES (original)
> +++ trunk/CHANGES 2008-04-04 15:31:07 EDT (Fri, 04 Apr 2008)
> @@ -50,6 +50,11 @@
>  - _details() - pass arbitrary values from test run sections
> to the mpi details section, indexed by string
>  
> +- Allow mpi_get_name, test_get_name, and test_build_name fields to
> +  accept the special value "all", meaning that they'll use all
> +  corresponding sections that are found (vs. needing to list every
> +  section explicitly)
> +
>  - Added export for MTT_TEST_EXECUTABLE, may be used for clean up after
>mpi process : pkill -9 $MTT_TEST_EXECUTABLE
>  
> 
> Modified: trunk/lib/MTT/MPI/Install.pm
> ==
> --- trunk/lib/MTT/MPI/Install.pm  (original)
> +++ trunk/lib/MTT/MPI/Install.pm  2008-04-04 15:31:07 EDT (Fri, 04 Apr 
> 2008)
> @@ -199,7 +199,8 @@
>  
>  # This is only warning about the INI file; we'll see
>  # if we find meta data for the MPI get later
> -if (!$ini_full->SectionExists("mpi get: $mpi_get_name")) {
> +if ($mpi_get_name ne "all" &&
> +!$ini_full->SectionExists("mpi get: $mpi_get_name")) {
>  Warning("Warning: MPI Get section \"$mpi_get_name\" does 
> not seem to exist in the INI file\n");
>  }
>  
> @@ -207,7 +208,8 @@
>  # skip it.  Don't issue a warning because command line
>  # parameters may well have dictated to skip this MPI
>  # get section.
> -if (!exists($MTT::MPI::sources->{$mpi_get_name})) {
> +if ($mpi_get_name ne "all" && 
> +!exists($MTT::MPI::sources->{$mpi_get_name})) {
>  Debug("Have no sources for MPI Get \"$mpi_get_name\", 
> skipping\n");
>  next;
>  }
> @@ -217,7 +219,8 @@
>  
>  # For each MPI source
>  foreach my $mpi_get_key (keys(%{$MTT::MPI::sources})) {
> -if ($mpi_get_key eq $mpi_get_name) {
> +if ($mpi_get_name eq "all" ||
> +$mpi_get_key eq $mpi_get_name) {
>  
>  # For each version of that source
>  my $mpi_get = $MTT::MPI::sources->{$mpi_get_key};
> 
> Modified: trunk/lib/MTT/Test/Build.pm
> ==
> --- trunk/lib/MTT/Test/Build.pm   (original)
> +++ trunk/lib/MTT/Test/Build.pm   2008-04-04 15:31:07 EDT (Fri, 04 Apr 
> 2008)
> @@ -120,7 +120,8 @@
>  
>  # This is only warning about the INI file; we'll see
>  # if we find meta data for the test get later
> -if (!$ini_full->SectionExists("test get: $test_get_name")) {
> +if ($test_get_name ne "all" &&
> +!$ini_full->SectionExists("test get: $test_get_name")) {
>  Warning("Test Get section \"$test_get_name\" does not 
> seem to exist in the INI file\n");
>  }
>  
> @@ -128,14 +129,16 @@
>  # skip it.  Don't issue a warning because command line
>  # parameters may well have dictated to skip this
>  # section.
> -if (!exists($MTT::Test::sources->{$test_get_name})) {
> +if ($test_get_name ne "all" &&
> +!exists($MTT::Test::sources->{$test_get_name})) {
>  Debug("Have no sources for Test Get \"$test_get_name\", 
> skipping\n");
>  next;
>  }
>  
>  # Find the matching test source
>  foreach my $test_get_key (keys(%{$MTT::Test::sources})) {
> -if ($test_get_key eq $test_get_name) {
> +if ($test_get_name eq "all" ||
> +$test_get_key eq $test_get_name) {
>  

Re: [MTT devel] [MTT svn] svn:mtt-svn r1164

[Jeff Squyres]

On Mar 20, 2008, at 12:19 PM, Ethan Mallove wrote:

I sense this "do_not_run" stuff could be useful to me, but
I'm not sure. Can you give a simple use case for
"do_not_run"? Could "do_not_run" be achieved by just
commenting out the INI lines that pertain to a certain
group? E.g., do these "#" comments ...

 # simple_really_slow:tests = src/MPI_Isend_flood_c src/ 
MPI_Send_flood_c


No, because if an upper-level _executables() found  
MPI_*end_flood_c, then they'll run in that section (vs. the  
simple_really_slow subsection).  The intent was to have a mechanism to  
specify: here's some tests, exclude them from all other sections, but  
*don't* run them.




 # simple_really_slow:pass = (_wifexited(),  
(_wexitstatus(), 0))

 # simple_really_slow:exclusive = 1
 # simple_really_slow:timeout = (90, (3, _np()))

... have the same effect as this?

 simple_really_slow:do_not_run = 1


Instead, you'd have:

simple_really_slow:tests = src/MPI_Isend_flood_c src/MPI_Send_flood_c
simple_really_slow:do_not_run = 1

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1164

[Ethan Mallove]

I sense this "do_not_run" stuff could be useful to me, but
I'm not sure. Can you give a simple use case for
"do_not_run"? Could "do_not_run" be achieved by just
commenting out the INI lines that pertain to a certain
group? E.g., do these "#" comments ...

  # simple_really_slow:tests = src/MPI_Isend_flood_c src/MPI_Send_flood_c
  # simple_really_slow:pass = (_wifexited(), (_wexitstatus(), 
0))
  # simple_really_slow:exclusive = 1
  # simple_really_slow:timeout = (90, (3, _np()))

... have the same effect as this?

  simple_really_slow:do_not_run = 1

-Ethan


On Thu, Mar/20/2008 10:03:50AM, jsquy...@osl.iu.edu wrote:
> Author: jsquyres
> Date: 2008-03-20 10:03:50 EDT (Thu, 20 Mar 2008)
> New Revision: 1164
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1164
> 
> Log:
> Added parameter to Simple specify module: do_not_run.  If set to 1 for
> a specific subsection (and the subsection is exclusive), then those
> tests will not be run.  Handy for explicitly specifying tests that
> exist by should ''not'' be run.
> 
> Text files modified: 
>trunk/CHANGES|12 +---  
>   
>trunk/lib/MTT/Test/Specify/Simple.pm |14 ++
>   
>2 files changed, 23 insertions(+), 3 deletions(-)
> 
> Modified: trunk/CHANGES
> ==
> --- trunk/CHANGES (original)
> +++ trunk/CHANGES 2008-03-20 10:03:50 EDT (Thu, 20 Mar 2008)
> @@ -168,7 +168,13 @@
>- before_make_install
>- after_make_install
>  
> -- include_file(s) INI parameter - just like pre-processor "#include" 
> directives
> +- include_file(s) INI parameter - just like pre-processor "#include"
> +  directives
>  
> -- trivial_tests_languages - comma separated list of languages to use for 
> trivial 
> -  tests (default: "c,c++,f77,f90").
> +- trivial_tests_languages - comma separated list of languages to use
> +  for trivial tests (default: "c,c++,f77,f90").
> +
> +- added parameter to Simple specify module: do_not_run.  If set to 1
> +  for a specific subsection (and the subsection is exclusive), then
> +  those tests will not be run.  Handy for explicitly specifying tests
> +  that exist but should *not* be run.
> 
> Modified: trunk/lib/MTT/Test/Specify/Simple.pm
> ==
> --- trunk/lib/MTT/Test/Specify/Simple.pm  (original)
> +++ trunk/lib/MTT/Test/Specify/Simple.pm  2008-03-20 10:03:50 EDT (Thu, 
> 20 Mar 2008)
> @@ -62,6 +62,7 @@
>  # Now go through and see if any of the tests are marked as
>  # "exclusive". If they are, remove those tests from all other
>  # groups.
> +my @groups_to_delete;
>  foreach my $group (keys %$params) {
>  # If this group is marked as exclusive, remove each of its
>  # tests from all other groups
> @@ -85,6 +86,19 @@
>  }
>  }
>  }
> +
> +# After we've performed the exclusivity filter, if the tests
> +# are marked as "do_not_run", then delete this group (it's a
> +# way of specifying tests to *not* run).  Don't delete them
> +# now, it may (will?) screw up the outter loop's "foreach".
> +if ($params->{$group}->{do_not_run}) {
> +push(@groups_to_delete, $group);
> +}
> +}
> +
> +# Delete all the groups that were marked
> +foreach my $t (@groups_to_delete) {
> +delete $params->{$t};
>  }
>  
>  # Now go through those groups and make the final list of tests to pass
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn

Re: [MTT devel] [MTT svn] svn:mtt-svn r1163

[Ethan Mallove]

On Mon, Mar/17/2008 11:01:11AM, Jeff Squyres wrote:
> Ethan --
> 
> Was there a case where the trivial plugin was not correctly detecting  
> what language bindings to compile against?
> 

The motivation is different than wanting to override the
language binding detection. Sometimes I *only* want to do
C++ tests (e.g., C++ STL sanity testing).

  [test-build-trivial]
  test_get = trivial
  save_stdout_on_success = 1
  merge_stdout_stderr = 1
  stderr_save_lines = 100
  module = Trivial

  [Test build: trivial]
  include_section = test-build-trivial

  # Only do C++ for STL section
  [Test build: trivial-stlport4]
  include_section = test-build-trivial
  trivial_tests_languages = c++
  trivial_tests_cflags = -library=stlport4

> (don't forget to add the new fields to the wiki)

Gotta copy *everything* in the CHANGES file to the wiki at some point!

-Ethan


> 
> 
> On Mar 17, 2008, at 10:55 AM, emall...@osl.iu.edu wrote:
> 
> > Author: emallove
> > Date: 2008-03-17 10:55:36 EDT (Mon, 17 Mar 2008)
> > New Revision: 1163
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1163
> >
> > Log:
> > * Added `trivial_tests_languages` to `Build/Trivial.pm`
> > * Added `clustertools_package_basedir` to
> >   `Install/ClusterTools.pm`
> > * Need to add whitespace to command in `Common/SCM.pm`
> >
> > Text files modified:
> >   trunk/CHANGES | 3 +++
> >   trunk/lib/MTT/Common/SCM.pm   |10 +-
> >   trunk/lib/MTT/MPI/Install/ClusterTools.pm | 9 ++---
> >   trunk/lib/MTT/Test/Build/Trivial.pm   |22 + 
> > +
> >   4 files changed, 32 insertions(+), 12 deletions(-)
> >
> > Modified: trunk/CHANGES
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > ==
> > --- trunk/CHANGES   (original)
> > +++ trunk/CHANGES   2008-03-17 10:55:36 EDT (Mon, 17 Mar 2008)
> > @@ -169,3 +169,6 @@
> >   - after_make_install
> >
> > - include_file(s) INI parameter - just like pre-processor "#include"  
> > directives
> > +
> > +- trivial_tests_languages - comma separated list of languages to  
> > use for trivial
> > +  tests (default: "c,c++,f77,f90").
> >
> > Modified: trunk/lib/MTT/Common/SCM.pm
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > ==
> > --- trunk/lib/MTT/Common/SCM.pm (original)
> > +++ trunk/lib/MTT/Common/SCM.pm 2008-03-17 10:55:36 EDT (Mon, 17 Mar  
> > 2008)
> > @@ -266,11 +266,11 @@
> >
> > # Append arguments to commands
> > # EAM: move to SVN.pm
> > -$command .= " $command_arguments "   if ($command_arguments);
> > -$command .= "-r $r "if ($r);
> > -$command .= "--username $username " if ($username);
> > -$command .= "--password $password " if ($password);
> > -$command .= "--no-auth-cache "  if ("0" eq $password_cache);
> > +$command .= " $command_arguments "  if ($command_arguments);
> > +$command .= " -r $r "if ($r);
> > +$command .= " --username $username " if ($username);
> > +$command .= " --password $password " if ($password);
> > +$command .= " --no-auth-cache "  if ("0" eq $password_cache);
> >
> > $subcommand .= " $subcommand_arguments "
> > if ($subcommand_arguments);
> >
> > Modified: trunk/lib/MTT/MPI/Install/ClusterTools.pm
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > = 
> > ==
> > --- trunk/lib/MTT/MPI/Install/ClusterTools.pm   (original)
> > +++ trunk/lib/MTT/MPI/Install/ClusterTools.pm   2008-03-17 10:55:36  
> > EDT (Mon, 17 Mar 2008)
> > @@ -31,6 +31,7 @@
> > my $release_number;
> > my $product_version;
> > my $package_name_prefix;
> > +my $package_basedir;
> >
> > sub Install {
> > my ($ini, $section, $config) = @_;
> > @@ -74,6 +75,8 @@
> > $release_number = Value($ini, $section, "clustertools_release");
> > $product_version = Value($ini, $section,  
> > "clustertools_product_version");
> > $package_name_prefix = Value($ini, $section,  
> > "clustertools_package_name_prefix");
> > +$package_basedir = Value($ini, $section,  
> > "clustertools_package_basedir");
> > +$package_basedir = $package_basedir ? $package_basedir : "/opt";
> >
> > # Grab the internal repository revision number
> > my $internal_r_number = $config->{module_data}->{r};
> > @@ -218,11 +221,11 @@
> >
> > # Setup the Installer, if we pointed at one
> > my $installer_dir;
> > -my $installer_dir_src = basename($installer_hg_url);
> > if ($installer_hg_url) {
> >
> > MTT::Module::Run("MTT::Common::SCM::Mercurial", "Checkout",  
> > "hg clone $installer_hg_url", $installer_hg_url);
> >
> > +my $installer_dir_src = basename($installer_hg_url);
> > MTT::DoCommand::Pushdir($installer_dir_src);
> >
> > # Build the Install_Utilities 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1163

[Jeff Squyres]

Ethan --

Was there a case where the trivial plugin was not correctly detecting  
what language bindings to compile against?


(don't forget to add the new fields to the wiki)


On Mar 17, 2008, at 10:55 AM, emall...@osl.iu.edu wrote:


Author: emallove
Date: 2008-03-17 10:55:36 EDT (Mon, 17 Mar 2008)
New Revision: 1163
URL: https://svn.open-mpi.org/trac/mtt/changeset/1163

Log:
* Added `trivial_tests_languages` to `Build/Trivial.pm`
* Added `clustertools_package_basedir` to
  `Install/ClusterTools.pm`
* Need to add whitespace to command in `Common/SCM.pm`

Text files modified:
  trunk/CHANGES | 3 +++
  trunk/lib/MTT/Common/SCM.pm   |10 +-
  trunk/lib/MTT/MPI/Install/ClusterTools.pm | 9 ++---
  trunk/lib/MTT/Test/Build/Trivial.pm   |22 + 
+

  4 files changed, 32 insertions(+), 12 deletions(-)

Modified: trunk/CHANGES
= 
= 
= 
= 
= 
= 
= 
= 
==

--- trunk/CHANGES   (original)
+++ trunk/CHANGES   2008-03-17 10:55:36 EDT (Mon, 17 Mar 2008)
@@ -169,3 +169,6 @@
  - after_make_install

- include_file(s) INI parameter - just like pre-processor "#include"  
directives

+
+- trivial_tests_languages - comma separated list of languages to  
use for trivial

+  tests (default: "c,c++,f77,f90").

Modified: trunk/lib/MTT/Common/SCM.pm
= 
= 
= 
= 
= 
= 
= 
= 
==

--- trunk/lib/MTT/Common/SCM.pm (original)
+++ trunk/lib/MTT/Common/SCM.pm	2008-03-17 10:55:36 EDT (Mon, 17 Mar  
2008)

@@ -266,11 +266,11 @@

# Append arguments to commands
# EAM: move to SVN.pm
-$command .= " $command_arguments "   if ($command_arguments);
-$command .= "-r $r "if ($r);
-$command .= "--username $username " if ($username);
-$command .= "--password $password " if ($password);
-$command .= "--no-auth-cache "  if ("0" eq $password_cache);
+$command .= " $command_arguments "  if ($command_arguments);
+$command .= " -r $r "if ($r);
+$command .= " --username $username " if ($username);
+$command .= " --password $password " if ($password);
+$command .= " --no-auth-cache "  if ("0" eq $password_cache);

$subcommand .= " $subcommand_arguments "
if ($subcommand_arguments);

Modified: trunk/lib/MTT/MPI/Install/ClusterTools.pm
= 
= 
= 
= 
= 
= 
= 
= 
==

--- trunk/lib/MTT/MPI/Install/ClusterTools.pm   (original)
+++ trunk/lib/MTT/MPI/Install/ClusterTools.pm	2008-03-17 10:55:36  
EDT (Mon, 17 Mar 2008)

@@ -31,6 +31,7 @@
my $release_number;
my $product_version;
my $package_name_prefix;
+my $package_basedir;

sub Install {
my ($ini, $section, $config) = @_;
@@ -74,6 +75,8 @@
$release_number = Value($ini, $section, "clustertools_release");
$product_version = Value($ini, $section,  
"clustertools_product_version");
$package_name_prefix = Value($ini, $section,  
"clustertools_package_name_prefix");
+$package_basedir = Value($ini, $section,  
"clustertools_package_basedir");

+$package_basedir = $package_basedir ? $package_basedir : "/opt";

# Grab the internal repository revision number
my $internal_r_number = $config->{module_data}->{r};
@@ -218,11 +221,11 @@

# Setup the Installer, if we pointed at one
my $installer_dir;
-my $installer_dir_src = basename($installer_hg_url);
if ($installer_hg_url) {

MTT::Module::Run("MTT::Common::SCM::Mercurial", "Checkout",  
"hg clone $installer_hg_url", $installer_hg_url);


+my $installer_dir_src = basename($installer_hg_url);
MTT::DoCommand::Pushdir($installer_dir_src);

# Build the Install_Utilities (OMPIompiat package)
@@ -667,7 +670,7 @@
# Write a pkginfo file for the specified package.
# To be passed to the prototype file.
sub _write_pkginfo_file {
-my ($name, $short_name, $desc) = @_;
+my ($name, $short_name, $desc, $package_basedir) = @_;
Debug("_write_pkginfo_file: got @_\n");

my $pkgvers;
@@ -699,7 +702,7 @@
PKG=\"$name\"
NAME=\"$short_name\"
VERSION=\"$release_number\"
-BASEDIR=\"/opt\"
+BASEDIR=\"$package_basedir\"
ARCH=\"ISA\"
SUNW_PRODVERS=\"$product_version\"
SUNW_PRODNAME=\"Open MPI\"

Modified: trunk/lib/MTT/Test/Build/Trivial.pm
= 
= 
= 
= 
= 
= 
= 
= 
==

--- trunk/lib/MTT/Test/Build/Trivial.pm (original)
+++ trunk/lib/MTT/Test/Build/Trivial.pm	2008-03-17 10:55:36 EDT  
(Mon, 17 Mar 2008)

@@ -55,10 +55,24 @@
   "trivial_tests_cflags");
my $fflags = Value($ini, $config->{full_section_name},
   "trivial_tests_fflags");
+my $languages = Value($ini, $config->{full_section_name},
+   "trivial_tests_languages");
+
+# Default to running *all* flavors of trivial tests
+if 

Re: [MTT devel] MTT Visualization

[Ethan Mallove]

On Fri, Jan/11/2008 12:49:50PM, Jeff Squyres wrote:
> On Jan 10, 2008, at 10:29 AM, Josh Hursey wrote:
> 
> > I met with Joseph Cottam (Grad student in my lab at IU) yesterday
> > about MTT visualization. He is working on some new visualization
> > techniques and wants to apply them to the MTT dataset.
> 
> Awesome.
> 
> > Since we are ramping up to a v1.3 release we want to visualization to
> > support this effort. So we want to make sure that the visualization
> > will meet the development community's needs. We should probably ask
> > the devel-core list, but I thought I would start some of the
> > discussion here to make sure I am asking the right questions of the
> > group.
> 
> Sounds reasonable.
> 
> After a first go-round here, we might want to have a conversation with  
> the OMPI RM's to get their input - that would still be a small group  
> to get targeted feedback on these questions.
> 
> > To start I have some basic questions:
> >  - How does Open MPI determine that it is stable enough to release?
> 
> I personally have a Magic 8 Ball on my desk that I consult frequently  
> for questions like this.  ;-)
> 
> It's a mix of many different metrics, actually:
> 
> - stuff unrelated to MTT results:
> - how many trac tickets are open against that release and do we care
> - how urgent are the bug fixes that are included
> - external requirements (e.g., get an OMPI release out to meet the  
> OFED release schedule)
> - ...and probably others
> - related to MTT results
> - "good" coverage on platforms (where "platform" = host arch, OS,  
> OS version, compiler, compiler version, MCA params, interconnect, and  
> scheduler -- note that some of these are orthogonal from each other...)
> - the only failures and timeouts we have are a) repeatable, b)  
> consistent across multiple organizations (if relevant), and deemed to  
> be acceptable
> 
> >  - What dimensions of testing are most/least important (i.e.,
> > platforms, compilers, feature sets, scale, ...)?
> 
> This is a hard question.  :-\  I listed several dimensions above:
> 
> - host architecture
> - OS
> - OS version
> - compiler
> - compiler version
> - MCA parameters used
> - interconnect
> - scheduler
> 
> Here's some more:
> 
> - number of processes tested
> - layout of processes (by node, by proc, ...etc.)
> 
> I don't quite know how to order those in terms of priority.  :-\
> 
> >  - What other questions would be useful to answer with regard to
> > testing (thinking completely outside of the box)?
> >* Example: Are we testing a specific platform/configuration set
> > too much/too little?
> 
> This is a great question.
> 
> I would love to be able to configure this question -- e.g., are we  
> testing some MCA params too much/too little.
> 
> The performance stuff can always be visualized better, especially over  
> time. One idea is expressed in https://svn.open-mpi.org/trac/mtt/ticket/330 
> .
> 
> I also very much like the ideas in 
> https://svn.open-mpi.org/trac/mtt/ticket/236 
>   and https://svn.open-mpi.org/trac/mtt/ticket/302 (302 is not  
> expressed as a visualization issue, but it could be -- you can imagine  
> a tree-based display showing the relationships between phase results,  
> perhaps even incorporated with a timeline -- that would be awesome).
> 
> Here's a whacky idea -- can our MTT data be combined with SCM data  
> (SVN, in this case) to answer questions like:
> 
> - what parts of the code are the most troublesome?  i.e., when this  
> part of the code changes, these tests tend to break
> 
> - what tests seem to be related to what parts of the OMPI code base?
> 
> - who / what SVN commit(s) seemed to cause specific tests to break?
> 
> (this seems like a longer-term set of questions, but I thought I'd  
> bring it up...)


I like this idea :-) 

A level of redirection missing to do this is keying SVN r
numbers to files modified. We also need to be able to
somehow track *new* failures (see
https://svn.open-mpi.org/trac/mtt/ticket/70). E.g., "was it
*this* revision that broke test xyz or was it an older one?"

-Ethan


> 
> >  - Other questions you think we should pose to the group?
> >
> > We are currently feeling out the domain of possibilities, but hope to
> > start doing some sketching some ideas in another week or so. This work
> > should proceed fairly quickly since we are targeting a paper about
> > this for the ACM Symposium on Software Visualization 
> > (http://www.softvis.org/
> > ) which is due in early April. How is that for expecting success :)
> 
> Awesome.
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] MTT Visualization

[Jeff Squyres]

On Jan 10, 2008, at 10:29 AM, Josh Hursey wrote:


I met with Joseph Cottam (Grad student in my lab at IU) yesterday
about MTT visualization. He is working on some new visualization
techniques and wants to apply them to the MTT dataset.


Awesome.


Since we are ramping up to a v1.3 release we want to visualization to
support this effort. So we want to make sure that the visualization
will meet the development community's needs. We should probably ask
the devel-core list, but I thought I would start some of the
discussion here to make sure I am asking the right questions of the
group.


Sounds reasonable.

After a first go-round here, we might want to have a conversation with  
the OMPI RM's to get their input - that would still be a small group  
to get targeted feedback on these questions.



To start I have some basic questions:
 - How does Open MPI determine that it is stable enough to release?


I personally have a Magic 8 Ball on my desk that I consult frequently  
for questions like this.  ;-)


It's a mix of many different metrics, actually:

- stuff unrelated to MTT results:
   - how many trac tickets are open against that release and do we care
   - how urgent are the bug fixes that are included
   - external requirements (e.g., get an OMPI release out to meet the  
OFED release schedule)

   - ...and probably others
- related to MTT results
   - "good" coverage on platforms (where "platform" = host arch, OS,  
OS version, compiler, compiler version, MCA params, interconnect, and  
scheduler -- note that some of these are orthogonal from each other...)
   - the only failures and timeouts we have are a) repeatable, b)  
consistent across multiple organizations (if relevant), and deemed to  
be acceptable



 - What dimensions of testing are most/least important (i.e.,
platforms, compilers, feature sets, scale, ...)?


This is a hard question.  :-\  I listed several dimensions above:

- host architecture
- OS
- OS version
- compiler
- compiler version
- MCA parameters used
- interconnect
- scheduler

Here's some more:

- number of processes tested
- layout of processes (by node, by proc, ...etc.)

I don't quite know how to order those in terms of priority.  :-\


 - What other questions would be useful to answer with regard to
testing (thinking completely outside of the box)?
   * Example: Are we testing a specific platform/configuration set
too much/too little?


This is a great question.

I would love to be able to configure this question -- e.g., are we  
testing some MCA params too much/too little.


The performance stuff can always be visualized better, especially over  
time. One idea is expressed in https://svn.open-mpi.org/trac/mtt/ticket/330 
. 

I also very much like the ideas in https://svn.open-mpi.org/trac/mtt/ticket/236 
 and https://svn.open-mpi.org/trac/mtt/ticket/302 (302 is not  
expressed as a visualization issue, but it could be -- you can imagine  
a tree-based display showing the relationships between phase results,  
perhaps even incorporated with a timeline -- that would be awesome).


Here's a whacky idea -- can our MTT data be combined with SCM data  
(SVN, in this case) to answer questions like:


- what parts of the code are the most troublesome?  i.e., when this  
part of the code changes, these tests tend to break


- what tests seem to be related to what parts of the OMPI code base?

- who / what SVN commit(s) seemed to cause specific tests to break?

(this seems like a longer-term set of questions, but I thought I'd  
bring it up...)



 - Other questions you think we should pose to the group?

We are currently feeling out the domain of possibilities, but hope to
start doing some sketching some ideas in another week or so. This work
should proceed fairly quickly since we are targeting a paper about
this for the ACM Symposium on Software Visualization (http://www.softvis.org/
) which is due in early April. How is that for expecting success :)


Awesome.

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT Visualization

[Ethan Mallove]

Woo hoo!

The reporter has been much much more useful since the DB
optimizations, though I wonder if in the next batch of
changes we could also have #296 (a few more columns are
needed)? (Not really related to visibility, but I thought
I'd speak my mind while there seems to be another round of
database stuff going on :-))

Stats visualization would be interesting on virtually any
column. Particularly combinations of these:

  * runtime_parameters
  * configure_options (broken down by individual options)
  * bitness
  * threadedness
  * np
  * platform
  * hardware

How much focus will be on the interface for generating
whatever visualization versus creating more static pictures?
It would be nice to have a flexible dashboard of some kind
that would allow us to generate ad hoc visualizations on any
combination of database columns.

-Ethan


On Thu, Jan/10/2008 10:29:32AM, Josh Hursey wrote:
> I met with Joseph Cottam (Grad student in my lab at IU)
> yesterday  about MTT visualization. He is working on some
> new visualization  techniques and wants to apply them to
> the MTT dataset.
> 
> Since we are ramping up to a v1.3 release we want to
> visualization to  support this effort. So we want to make
> sure that the visualization  will meet the development
> community's needs. We should probably ask  the devel-core
> list, but I thought I would start some of the  discussion
> here to make sure I am asking the right questions of the
> group.
> 
> To start I have some basic questions:
>   - How does Open MPI determine that it is stable enough to release?
>   - What dimensions of testing are most/least important (i.e.,  
> platforms, compilers, feature sets, scale, ...)?
>   - What other questions would be useful to answer with regard to  
> testing (thinking completely outside of the box)?
> * Example: Are we testing a specific platform/configuration set  
> too much/too little?
>   - Other questions you think we should pose to the group?
> 
> We are currently feeling out the domain of possibilities,
> but hope to  start doing some sketching some ideas in
> another week or so. This work  should proceed fairly
> quickly since we are targeting a paper about  this for the
> ACM Symposium on Software Visualization
> (http://www.softvis.org/ ) which is due in early April.
> How is that for expecting success :)
> 
> Cheers,
> Josh
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] mtt reporter popups broken

[Jeff Squyres]

Perfect -- I can confirm that it all works again.

Thanks!


On Jan 10, 2008, at 12:53 PM, Josh Hursey wrote:


Fixed. This was a latent bug that my commit highlighted, but I didn't
check for (Sorry). If mtt_body_html_suffix was non-null then the
popups would not work. I just fixed this in r1133.

-- Josh

On Jan 10, 2008, at 12:26 PM, Jeff Squyres wrote:


I notice that I can no longer get the MTT preferences/advanced popups
anymore in Safari.  They were working earlier this morning -- did
Josh's commit break something, perchance?

--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt reporter popups broken

[Josh Hursey]

Fixed. This was a latent bug that my commit highlighted, but I didn't  
check for (Sorry). If mtt_body_html_suffix was non-null then the  
popups would not work. I just fixed this in r1133.


-- Josh

On Jan 10, 2008, at 12:26 PM, Jeff Squyres wrote:


I notice that I can no longer get the MTT preferences/advanced popups
anymore in Safari.  They were working earlier this morning -- did
Josh's commit break something, perchance?

--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] [MTT svn] svn:mtt-svn r1094 (for review)

[Ethan Mallove]

Josh,

Before this gets committed to the live submit.php, can you
look at this? It should only effect the "environment" field.

Thanks,
Ethan


On Thu, Nov/01/2007 03:30:26PM, emall...@osl.iu.edu wrote:
> Author: emallove
> Date: 2007-11-01 15:30:26 EDT (Thu, 01 Nov 2007)
> New Revision: 1094
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1094
> 
> Log:
>  * "environment" field was not getting inserted due to
>a faulty `isset()` check
>  * Added a second MTT DB admin
> 
> 
> Text files modified: 
>trunk/server/php/submit/index.php |28 +++- 
>
>1 files changed, 19 insertions(+), 9 deletions(-)
> 
> Modified: trunk/server/php/submit/index.php
> ==
> --- trunk/server/php/submit/index.php (original)
> +++ trunk/server/php/submit/index.php 2007-11-01 15:30:26 EDT (Thu, 01 Nov 
> 2007)
> @@ -235,6 +235,9 @@
>  
>  for($i = 0; $i < $n; $i++) {
>  
> +# The POST fields are enumerated starting at 1
> +$j = $i + 1;
> +
>  
>  # Select/Insert: performance
>  # Currently only support latency/bandwidth
> @@ -363,7 +366,7 @@
>  #
>  # Select/Insert: Environment
>  $results_idxs_hash['environment_id'] = 0;
> -if( isset($_POST['environment']) ) {
> +if( isset($_POST["environment_$j"]) ) {
>  $stmt_fields = array("environment");
>  
>  $stmt_values = array(get_scalar($param_set['environment'], $i) );
> @@ -567,6 +570,9 @@
>  }
>  
>  for($i = 0; $i < $n; $i++) {
> +
> +# The POST fields are enumerated starting at 1
> +$j = $i + 1;
>  
>  
>  # Select/Insert: test_build_compiler -> compiler
> @@ -629,7 +635,8 @@
>  #
>  # Select/Insert: Environment
>  $results_idxs_hash['environment_id'] = 0;
> -if( isset($_POST['environment']) ) {
> +
> +if( isset($_POST["environment_$j"]) ) {
>  $stmt_fields = array("environment");
>  
>  $stmt_values = array(get_scalar($param_set['environment'], $i) );
> @@ -1084,6 +1091,9 @@
>  
>  for($i = 0; $i < $n; $i++) {
>  
> +# The POST fields are enumerated starting at 1
> +$j = $i + 1;
> +
>  
>  # Select/Insert: compute_cluster
>  $stmt_fields = array("platform_name",
> @@ -1185,7 +1195,7 @@
>  #
>  # Select/Insert: Environment
>  $results_idxs_hash['environment_id'] = 0;
> -if( isset($_POST['environment']) ) {
> +if( isset($_POST["environment_$j"]) ) {
>  $stmt_fields = array("environment");
>  
>  $stmt_values = array(get_scalar($param_set['environment'], $i) );
> @@ -1767,8 +1777,8 @@
>  
>  $php_auth_user = $_SERVER['PHP_AUTH_USER'];
>  $user  = $_POST['email'];
> -#JJH$admin = 'ethan.mall...@sun.com';
> -$admin = 'jjhur...@open-mpi.org';
> +$admin1= 'jjhur...@open-mpi.org';
> +$admin2= 'ethan.mall...@sun.com';
>  $date  = date('r');
>  $phpversion= phpversion();
>  $boundary  = md5(time());
> @@ -1778,8 +1788,8 @@
>  $attachment = chunk_split(base64_encode(file_get_contents($filename)));
>  
>  $headers = << -From: $admin
> -Reply-To: $admin
> +From: $admin1
> +Reply-To: $admin1
>  Date: $date
>  X-Mailer: PHP v$phpversion
>  MIME-Version: 1.0
> @@ -1808,8 +1818,8 @@
>  if (preg_match("/\w+@\w+/", $user, $m))
>  mail($user, "MTT server error", $message, $headers);
>  
> -# Email the MTT database administrator
> -mail($admin, "MTT server error (user: $php_auth_user)", $message, 
> $headers);
> +# Email the MTT database administrator(s)
> +mail("$admin1, $admin2", "MTT server error (user: $php_auth_user)", 
> $message, $headers);
>  
>  # Whack the temp file
>  unlink($filename);
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn

Re: [MTT devel] MTT server error (user: uh)

[Jeff Squyres]

Agreed.  Mohamad -- can you fix?


On Oct 15, 2007, at 11:42 AM, Josh Hursey wrote:


I'm getting the following from 'uh'. The problem is that they supply
'-np' with no argument. The submit script is rejecting the submit, so
the database is fine. I think this is a user problem, but it kind of
looks like a client problem.

Thought I would post to see if anyone has any thoughts.

-- Josh

On Oct 15, 2007, at 11:37 AM, jjhur...@open-mpi.org wrote:



SQL QUERY: INSERT INTO test_run
(test_run_id, submit_id, compute_cluster_id,
mpi_install_compiler_id, mpi_get_id, mpi_install_configure_id,
mpi_install_id, test_suite_id, test_build_compiler_id,
test_build_id, test_name_id, performance_id, test_run_command_id,
np, full_command, description_id, start_timestamp, test_result,
trial, submit_timestamp, duration, environment_id, result_stdout,
result_stderr, result_message_id, merge_stdout_stderr, exit_value,
exit_signal, client_serial) VALUES
('25748604', '3227', '141', '37', '602', '567', '13687', '4',
'37', '60596', '700', '0', '2', '', 'mpirun  -np  --mca btl self,sm
--prefix /home/mschaara/mtt-testing/scratch/scratch2/installs/lAQN/
install src/IMB-MPI1 -npmin  PingPong', '0', 'Mon Oct 15 15:37:29
2007', '0', '1', DEFAULT, '0 seconds', '0',
' 
-

-
mpirun was unable to launch the specified application as it could
not find an executable:

Executable: btl
Node: shark01

while attempting to start process rank 0.
- 
-


3 additional processes aborted (not shown)
', '', '82', DEFAULT, '1', '-1', '10200')
SQL ERROR: ERROR:  invalid input syntax for integer: ""
SQL ERROR:


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT server error (user: uh)

[Josh Hursey]

I'm getting the following from 'uh'. The problem is that they supply  
'-np' with no argument. The submit script is rejecting the submit, so  
the database is fine. I think this is a user problem, but it kind of  
looks like a client problem.


Thought I would post to see if anyone has any thoughts.

-- Josh

On Oct 15, 2007, at 11:37 AM, jjhur...@open-mpi.org wrote:



SQL QUERY: INSERT INTO test_run
	(test_run_id, submit_id, compute_cluster_id,  
mpi_install_compiler_id, mpi_get_id, mpi_install_configure_id,  
mpi_install_id, test_suite_id, test_build_compiler_id,  
test_build_id, test_name_id, performance_id, test_run_command_id,  
np, full_command, description_id, start_timestamp, test_result,  
trial, submit_timestamp, duration, environment_id, result_stdout,  
result_stderr, result_message_id, merge_stdout_stderr, exit_value,  
exit_signal, client_serial) VALUES
	('25748604', '3227', '141', '37', '602', '567', '13687', '4',  
'37', '60596', '700', '0', '2', '', 'mpirun  -np  --mca btl self,sm  
--prefix /home/mschaara/mtt-testing/scratch/scratch2/installs/lAQN/ 
install src/IMB-MPI1 -npmin  PingPong', '0', 'Mon Oct 15 15:37:29  
2007', '0', '1', DEFAULT, '0 seconds', '0',  
'- 
-
mpirun was unable to launch the specified application as it could  
not find an executable:


Executable: btl
Node: shark01

while attempting to start process rank 0.
-- 


3 additional processes aborted (not shown)
', '', '82', DEFAULT, '1', '-1', '10200')
SQL ERROR: ERROR:  invalid input syntax for integer: ""
SQL ERROR:

Re: [MTT devel] [MTT svn] svn:mtt-svn r1066

[Jeff Squyres]

Ethan --

Don't forget to document all these new fields / funclets on the wiki...


On Sep 29, 2007, at 12:27 AM, emall...@osl.iu.edu wrote:


Author: emallove
Date: 2007-09-28 18:27:54 EDT (Fri, 28 Sep 2007)
New Revision: 1066
URL: https://svn.open-mpi.org/trac/mtt/changeset/1066

Log:
 * Added five `Noop.pm` modules (enables one to simply use the MTT  
as a run

   engine in conjunction with an ''MPI'' Details section)
 * Added `@PROGRAM_NAME@` as an INI predefined
 * Added `--clean-start` option to run MTT using a fresh scratch dir
 * Added `_details_simple_name()` funclet
 * Use `Util::does_hash_key_exist()` in `Test/Build.pm`
 * Typo (missing `$`) in `MPI/Install.pm`

Added:
   trunk/lib/MTT/MPI/Get/Noop.pm
   trunk/lib/MTT/MPI/Install/Noop.pm
   trunk/lib/MTT/Test/Build/Noop.pm
   trunk/lib/MTT/Test/Get/Noop.pm
   trunk/lib/MTT/Test/Specify/Noop.pm
Text files modified:
   trunk/client/mtt  |11 +++
   trunk/lib/MTT/INI.pm  | 7 +++
   trunk/lib/MTT/MPI/Install.pm  | 2 +-
   trunk/lib/MTT/Test/Build.pm   | 9 +
   trunk/lib/MTT/Test/Run.pm | 1 +
   trunk/lib/MTT/Values/Functions.pm | 5 +
   6 files changed, 30 insertions(+), 5 deletions(-)

Modified: trunk/client/mtt
== 


--- trunk/client/mtt(original)
+++ trunk/client/mtt2007-09-28 18:27:54 EDT (Fri, 28 Sep 2007)
@@ -117,6 +117,7 @@
 my $verbose_arg;
 my $no_execute_arg;
 my $force_arg;
+my $clean_start_arg;
 my $mpi_get_arg;
 my $mpi_install_arg;
 my $test_get_arg;
@@ -141,6 +142,7 @@
   "verbose|v" => \$verbose_arg,
   "no-execute|n" => \$no_execute_arg,
   "force" => \$force_arg,
+  "clean-start" => \$clean_start_arg,
   "mpi-get!" => \$mpi_get_arg,
   "mpi-install!" => \ 
$mpi_install_arg,

   "test-get!" => \$test_get_arg,
@@ -204,6 +206,8 @@
 --print-time|-p   Display the amount of time taken in  
each phase
 --force   Do steps even if they would not  
normally

   be executed
+--clean-start Clean the scratch directory from
+  past MTT invocations before running
 --[no-]trial  Use when testing your MTT client setup;
   results that are generated and  
submitted
   to the database are marked as  
\"trials\"

@@ -353,6 +357,13 @@
 my $runs_data_dir =
 MTT::Files::mkdir("$scratch_arg/$MTT::Defaults::System_config-> 
{runs_data_subdir}");


+# If requested, do a clean start
+if ($clean_start_arg) {
+MTT::DoCommand::Cmd("rm -rf $source_dir");
+MTT::DoCommand::Cmd("rm -rf $install_dir");
+MTT::DoCommand::Cmd("rm -rf $runs_data_dir");
+}
+
 # Load up all the MPI sources that this system has previously  
obtained

 MTT::MPI::LoadSources($source_dir)
 if ($mpi_get || $mpi_install || $test_build || $test_run ||  
$trim);


Modified: trunk/lib/MTT/INI.pm
== 


--- trunk/lib/MTT/INI.pm(original)
+++ trunk/lib/MTT/INI.pm2007-09-28 18:27:54 EDT (Fri, 28 Sep 2007)
@@ -247,6 +247,13 @@
 }
 }

+foreach my $section ($ini->Sections) {
+if (! defined($ini->val($section, "PROGRAM_NAME"))) {
+$ini->delval($section, "PROGRAM_NAME");
+$ini->newval($section, "PROGRAM_NAME", $0);
+}
+}
+
 return $ini;
 }


Added: trunk/lib/MTT/MPI/Get/Noop.pm
== 


--- (empty file)
+++ trunk/lib/MTT/MPI/Get/Noop.pm	2007-09-28 18:27:54 EDT (Fri, 28  
Sep 2007)

@@ -0,0 +1,36 @@
+#!/usr/bin/env perl
+#
+# Copyright (c) 2007 Sun Microsystems, Inc.  All rights reserved.
+# $COPYRIGHT$
+#
+# Additional copyrights may follow
+#
+# $HEADER$
+#
+
+package MTT::MPI::Get::Noop;
+my $package = __PACKAGE__;
+
+use strict;
+use MTT::Values;
+use Cwd;
+
+# 
--

+
+sub Get {
+my $ret;
+$ret->{version} = MTT::Values::RandomString(10);
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{have_new} = 1;
+$ret->{result_message} = "Success";
+$ret->{prepare_for_install} = "${package}::PrepareForInstall";
+return $ret;
+}
+
+# 
--

+
+sub PrepareForInstall {
+return cwd();
+}
+
+1;

Modified: trunk/lib/MTT/MPI/Install.pm
== 


--- trunk/lib/MTT/MPI/Install.pm(original)
+++ trunk/lib/MTT/MPI/Install.pm	2007-09-28 18:27:54 EDT (Fri, 28  
Sep 

Re: [MTT devel] MTT server error (user: cisco)

[Jeff Squyres]

It could be that I was hitting ctrl-C for a bunch of my tests.

Let's ignore unless it repeats; thanks for being diligent, though!


On Sep 22, 2007, at 10:56 AM, Josh Hursey wrote:


So I got two of these emails this morning. Any thoughts on why these
2 latency/bandwidth submissions are failing?

The problem is with:
SQL ERROR: ERROR:  malformed array literal:
"{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124.6 
9,

,
124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,7342. 
21

,15831.27,38853.38,77511.53,130775.23}"

More specifically it is probably the empty array entry about 11
entries into the array ("124.69,,124.81"). Could this be the result
of an error that was not caught properly?

-- josh

On Sep 22, 2007, at 8:23 AM, jjhur...@open-mpi.org wrote:



SQL QUERY: INSERT INTO latency_bandwidth
(latency_bandwidth_id, message_size, latency_min, latency_avg,
latency_max, bandwidth_min, bandwidth_avg, bandwidth_max) VALUES
('107215',
'{0,1,2,4,8,16,32,64,128,256,19024,512,1024,2048,4096,8192,16384,3276 
8

,65536,131072,262144,524288,1048576,2097152,4194304}',
'{135.69,124.04,124.28,124.20,124.40,124.27,124.17,124.29,124.26,124. 
2
9,5,124.40,140.17,185.32,187.23,307.94,418.52,741.56,1568.65,3537.90, 
7

314.73,15428.99,37013.45,68857.91,94.49}',
'{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124. 
6
9,,124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,7 
3

42.21,15831.27,38853.38,77511.53,130775.23}',
'{137.07,125.04,125.02,125.07,125.17,124.93,124.93,151.80,125.04,125. 
0
8,,125.29,141.53,187.11,187.89,310.59,419.98,744.73,1580.83,3546.58,7 
3

71.56,16254.56,40249.37,87200.25,149653.98}', DEFAULT,
'{0.00,0.03,0.06,0.12,0.24,0.49,0.98,1.61,3.90,7.81,,15.59,27.60,41.7 
5

,
83.16,100.61,148.82,167.85,158.14,140.98,135.66,123.04,99.38,91.74,10 
6

.91}', DEFAULT)
SQL ERROR: ERROR:  malformed array literal:
"{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124. 
6
9,,124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,7 
3

42.21,15831.27,38853.38,77511.53,130775.23}"
SQL ERROR:


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT server error (user: cisco)

[Josh Hursey]

So I got two of these emails this morning. Any thoughts on why these  
2 latency/bandwidth submissions are failing?


The problem is with:
SQL ERROR: ERROR:  malformed array literal:  
"{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124.69, 
, 
124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,7342.21 
,15831.27,38853.38,77511.53,130775.23}"


More specifically it is probably the empty array entry about 11  
entries into the array ("124.69,,124.81"). Could this be the result  
of an error that was not caught properly?


-- josh

On Sep 22, 2007, at 8:23 AM, jjhur...@open-mpi.org wrote:



SQL QUERY: INSERT INTO latency_bandwidth
	(latency_bandwidth_id, message_size, latency_min, latency_avg,  
latency_max, bandwidth_min, bandwidth_avg, bandwidth_max) VALUES
	('107215',  
'{0,1,2,4,8,16,32,64,128,256,19024,512,1024,2048,4096,8192,16384,32768 
,65536,131072,262144,524288,1048576,2097152,4194304}',  
'{135.69,124.04,124.28,124.20,124.40,124.27,124.17,124.29,124.26,124.2 
9,5,124.40,140.17,185.32,187.23,307.94,418.52,741.56,1568.65,3537.90,7 
314.73,15428.99,37013.45,68857.91,94.49}',  
'{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124.6 
9,,124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,73 
42.21,15831.27,38853.38,77511.53,130775.23}',  
'{137.07,125.04,125.02,125.07,125.17,124.93,124.93,151.80,125.04,125.0 
8,,125.29,141.53,187.11,187.89,310.59,419.98,744.73,1580.83,3546.58,73 
71.56,16254.56,40249.37,87200.25,149653.98}', DEFAULT,  
'{0.00,0.03,0.06,0.12,0.24,0.49,0.98,1.61,3.90,7.81,,15.59,27.60,41.75 
, 
83.16,100.61,148.82,167.85,158.14,140.98,135.66,123.04,99.38,91.74,106 
.91}', DEFAULT)
SQL ERROR: ERROR:  malformed array literal:  
"{136.36,124.59,124.58,124.61,124.74,124.52,124.57,137.98,124.62,124.6 
9,,124.81,140.91,186.13,187.51,309.31,419.06,743.33,1574.05,3542.81,73 
42.21,15831.27,38853.38,77511.53,130775.23}"

SQL ERROR:

Re: [MTT devel] MTT server error (user: sun)

[Josh Hursey]

I just implemented a 'fix' for this in which if the client submits a  
test_build or test_run without proper lineage instead of guessing  
(any mostly getting it wrong) we just point the result to an 'undef'  
row. This will allow us to more effectively track these going  
forward, and maybe even remove them if needed.


So I guess I wouldn't worry about it past this morning, but it is  
something we might want to ponder about a bit more by looking at how  
the reporter/submit scripts should deal with Already Installed  
scenarios.


-- Josh

On Sep 19, 2007, at 10:08 AM, Ethan Mallove wrote:


On Mon, Sep/17/2007 10:46:45AM, Josh Hursey wrote:

I've been getting quite a few errors from submit.php of
the below  form. It seems that the MPI Install that is
being referenced is not  valid. Could this is an 'already
installed' issue where the user is  trying to submit
results to the database without having submitted an
mpi_install?



You are correct. Some test reporter-less runs were done
before some runs that used the MTTDatabase reporter were
done. I guess the rule of thumb is to, from the outset, run
using --trial if you plan to eventually send to the
database.

-Ethan



-- Josh

On Sep 17, 2007, at 10:37 AM, jjhur...@open-mpi.org wrote:


find_mpi_install_id():
The following SELECT returned -1:
---
SELECT mpi_install_id
FROM mpi_install NATURAL JOIN
 mpi_get NATURAL JOIN
 compilerNATURAL JOIN
 compute_cluster NATURAL JOIN
 submit
WHERE
mpi_version = '1.2.4b1r16122M'  AND
mpi_name= 'developer'  AND
compiler_version= '4.1.2'  AND
hostname= 'fog01'  AND
mtt_client_version= '2.1devel'  AND
local_username= 'paklui'  AND
platform_name= 'fog01'
ORDER BY mpi_install_id DESC limit 1
---



___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] MTT server error (user: sun)

[Ethan Mallove]

On Mon, Sep/17/2007 10:46:45AM, Josh Hursey wrote:
> I've been getting quite a few errors from submit.php of
> the below  form. It seems that the MPI Install that is
> being referenced is not  valid. Could this is an 'already
> installed' issue where the user is  trying to submit
> results to the database without having submitted an
> mpi_install?


You are correct. Some test reporter-less runs were done
before some runs that used the MTTDatabase reporter were
done. I guess the rule of thumb is to, from the outset, run
using --trial if you plan to eventually send to the
database.

-Ethan

> 
> -- Josh
> 
> On Sep 17, 2007, at 10:37 AM, jjhur...@open-mpi.org wrote:
> 
> > find_mpi_install_id():
> > The following SELECT returned -1:
> > ---
> > SELECT mpi_install_id
> > FROM mpi_install NATURAL JOIN
> >  mpi_get NATURAL JOIN
> >  compilerNATURAL JOIN
> >  compute_cluster NATURAL JOIN
> >  submit
> > WHERE
> > mpi_version = '1.2.4b1r16122M'  AND
> > mpi_name= 'developer'  AND
> > compiler_version= '4.1.2'  AND
> > hostname= 'fog01'  AND
> > mtt_client_version= '2.1devel'  AND
> > local_username= 'paklui'  AND
> > platform_name= 'fog01'
> > ORDER BY mpi_install_id DESC limit 1
> > ---
> >
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] MTT server error (user: uh)

[Jeff Squyres]

That's weird; I don't know.

Mohamad: can you send a snipit of your INI file that sets up the  
value "run_random_hostname_patterns.pl"?  I'm curious to see how it's  
propagating down into the resource_manager value...



On Sep 18, 2007, at 8:43 PM, Josh Hursey wrote:


This is weird. How could the script "./wrappers/
run_random_hostname_patterns.pl" be submitted as the 'launcher' to
the database? I thought submit.php would only get valid launchers
from the client?

-- Josh

On Sep 18, 2007, at 8:05 PM, jjhur...@open-mpi.org wrote:



SQL QUERY: INSERT INTO test_run_command
(test_run_command_id, launcher, resource_mgr, parameters, network,
test_run_network_id) VALUES
('132', './wrappers/run_random_hostname_patterns.pl', 'slurm', '',
'', '2')
SQL ERROR: ERROR:  value too long for type character varying(16)
SQL ERROR:


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r998

[Jeff Squyres]

Ethan --

Could you show the use case that motivated this change?

Thanks.


On Sep 7, 2007, at 11:52 AM, emall...@osl.iu.edu wrote:


Author: emallove
Date: 2007-09-07 11:52:04 EDT (Fri, 07 Sep 2007)
New Revision: 998
URL: https://svn.open-mpi.org/trac/mtt/changeset/998

Log:
Escape the Perl regular expression quantifiers in
`::OMPI::find_network` (for test names such as
`mpic++`).

Text files modified:
   tmp/jms-new-parser/lib/MTT/Values/Functions/MPI/OMPI.pm | 3 +++
   1 files changed, 3 insertions(+), 0 deletions(-)

Modified: tmp/jms-new-parser/lib/MTT/Values/Functions/MPI/OMPI.pm
== 


--- tmp/jms-new-parser/lib/MTT/Values/Functions/MPI/OMPI.pm (original)
+++ tmp/jms-new-parser/lib/MTT/Values/Functions/MPI/OMPI.pm	 
2007-09-07 11:52:04 EDT (Fri, 07 Sep 2007)

@@ -98,6 +98,9 @@
 # Ignore argv[0]
 $str =~ s/^\s*\S+\s*(.+)$/\1/;

+# Escape the quantifiers (for test names such as "mpi2c++")
+$final =~ s/(\?|\*|\+|\{|\})/\\$1/g;
+
 # Ignore everything beyond $final
 $str =~ s/^(.+)\s*$final.+$/\1/;
 Debug("Examining: $str\n");
___
mtt-svn mailing list
mtt-...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT users] Test runs not getting into database

[Jeff Squyres]

Josh / Ethan --

Not getting a serial means that the client is not getting a value  
back from the server that it can parse into a serial.


Can you guys dig into this and see why the mtt dbdebug file that Tim  
has at the end of this message is not getting a serial?


Thanks...


On Sep 5, 2007, at 9:24 AM, Tim Prins wrote:


Here is the smallest one. Let me know if you need anything else.

Tim

Jeff Squyres wrote:
Can you send any one of those mtt database files?  We'll need to   
figure out if this is a client or a server problem.  :-(

On Sep 5, 2007, at 7:40 AM, Tim Prins wrote:

Hi,

BigRed has not gotten its test results into the database for a  
while.

This is running the ompi-core-testers branch. We run by passing the
results through the mtt-relay.

The mtt-output file has lines like:
*** WARNING: MTTDatabase did not get a serial; phases will be  
isolated

from each other in the reports

Reported to MTTDatabase: 1 successful submit, 0 failed submits

(total of 1 result)

I have the database submit files if they would help.

Thanks,

Tim

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users


$VAR1 = {
  'exit_signal_1' => -1,
  'duration_1' => '5 seconds',
  'mpi_version' => '1.3a1r16038',
  'trial' => 0,
  'mpi_install_section_name_1' => 'bigred 32 bit gcc',
  'client_serial' => undef,
  'hostname' => 's1c2b12',
  'result_stdout_1' => '/bin/rm -f *.o *~ PI* core IMB-IO  
IMB-EXT IMB-MPI1 exe_io exe_ext exe_mpi1

touch IMB_declare.h
touch exe_mpi1 *.c; rm -rf exe_io exe_ext
make MPI1 CPP=MPI1
make[1]: Entering directory `/N/ptl01/mpiteam/bigred/20070905- 
Wednesday/pb_0/installs/d7Ri/tests/imb/IMB_2.3/src\'

mpicc  -I.  -DMPI1 -O -c IMB.c
mpicc  -I.  -DMPI1 -O -c IMB_declare.c
mpicc  -I.  -DMPI1 -O -c IMB_init.c
mpicc  -I.  -DMPI1 -O -c IMB_mem_manager.c
mpicc  -I.  -DMPI1 -O -c IMB_parse_name_mpi1.c
mpicc  -I.  -DMPI1 -O -c IMB_benchlist.c
mpicc  -I.  -DMPI1 -O -c IMB_strgs.c
mpicc  -I.  -DMPI1 -O -c IMB_err_handler.c
mpicc  -I.  -DMPI1 -O -c IMB_g_info.c
mpicc  -I.  -DMPI1 -O -c IMB_warm_up.c
mpicc  -I.  -DMPI1 -O -c IMB_output.c
mpicc  -I.  -DMPI1 -O -c IMB_pingpong.c
mpicc  -I.  -DMPI1 -O -c IMB_pingping.c
mpicc  -I.  -DMPI1 -O -c IMB_allreduce.c
mpicc  -I.  -DMPI1 -O -c IMB_reduce_scatter.c
mpicc  -I.  -DMPI1 -O -c IMB_reduce.c
mpicc  -I.  -DMPI1 -O -c IMB_exchange.c
mpicc  -I.  -DMPI1 -O -c IMB_bcast.c
mpicc  -I.  -DMPI1 -O -c IMB_barrier.c
mpicc  -I.  -DMPI1 -O -c IMB_allgather.c
mpicc  -I.  -DMPI1 -O -c IMB_allgatherv.c
mpicc  -I.  -DMPI1 -O -c IMB_alltoall.c
mpicc  -I.  -DMPI1 -O -c IMB_sendrecv.c
mpicc  -I.  -DMPI1 -O -c IMB_init_transfer.c
mpicc  -I.  -DMPI1 -O -c IMB_chk_diff.c
mpicc  -I.  -DMPI1 -O -c IMB_cpu_exploit.c
mpicc   -o IMB-MPI1 IMB.o IMB_declare.o  IMB_init.o  
IMB_mem_manager.o IMB_parse_name_mpi1.o  IMB_benchlist.o  
IMB_strgs.o IMB_err_handler.o IMB_g_info.o  IMB_warm_up.o  
IMB_output.o IMB_pingpong.o IMB_pingping.o IMB_allreduce.o  
IMB_reduce_scatter.o IMB_reduce.o IMB_exchange.o IMB_bcast.o  
IMB_barrier.o IMB_allgather.o IMB_allgatherv.o IMB_alltoall.o  
IMB_sendrecv.o IMB_init_transfer.o  IMB_chk_diff.o IMB_cpu_exploit.o
make[1]: Leaving directory `/N/ptl01/mpiteam/bigred/20070905- 
Wednesday/pb_0/installs/d7Ri/tests/imb/IMB_2.3/src\'

',
  'mpi_name' => 'ompi-nightly-trunk',
  'number_of_results' => '1',
  'phase' => 'Test Build',
  'compiler_version_1' => '3.3.3',
  'exit_value_1' => 0,
  'result_message_1' => 'Success',
  'start_timestamp_1' => 'Wed Sep  5 04:16:52 2007',
  'compiler_name_1' => 'gnu',
  'suite_name_1' => 'imb',
  'test_result_1' => 1,
  'mtt_client_version' => '2.1devel',
  'fields' =>  
'compiler_name,compiler_version,duration,exit_signal,exit_value,mpi_ge 
t_section_name,mpi_install_id,mpi_install_section_name,mpi_name,mpi_ve 
rsion,phase,result_message,result_stdout,start_timestamp,suite_name,te 
st_result',

  'mpi_install_id' => undef,
  'platform_name' => 'IU_BigRed',
  'local_username' => 'mpiteam',
  'mpi_get_section_name_1' => 'ompi-nightly-trunk'
};
___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT users] Database submit error

[Josh Hursey]

Sounds good. Cleaning up now.

Cheers,
Josh

On Aug 31, 2007, at 1:38 PM, Jeff Squyres wrote:


No objections.  If the data is junk, just ditch it.

On Aug 31, 2007, at 12:47 PM, Josh Hursey wrote:


I was looking at the data from Monday Aug 27, 8 am to Tuesday Aug 28,
Noonish when this problem was occuring, and the data is mostly
invalid. We have test_builds pointing at the wrong test_suites. Since
this brings all of this data inso suspicion I'm going through and
flaging them all as 'trial'.

If you don't have any conflict, then I'd like to remove this data
alltogether from the database so the normalization tables can be
cleaned up.

Any objections to removing the set of data in the time range Monday
Aug 27, 8 am to Tuesday Aug 28, Noonish? it's about 8,000 test_runs
since most of the test runs were getting rejected during that time
period we are not losing any good data.

-- Josh


On Aug 28, 2007, at 10:27 AM, Josh Hursey wrote:


Short Version:
--
I just finished the fix, and the submit script is back up and
running.

This was a bug that arose in testing, but somehow did not get
propagated to the production database.

Long Version:
-
The new databases uses partition tables to archive test results. As
part of this there are some complex rules to mask the partition  
table

complexity from the users of the db. There was a bug in the insert
rule in which the 'id' of the submitted result (mpi_install,
test_build, and test_run) was a different value than expected since
the 'id' was not translated properly to the partition table setup.

The fix was to drop all rules and replace them with the correct
versions. The submit errors you saw below were caused by integrity
checks in the submit script that keep data from being submitted that
do not have a proper lineage (e.g., you cannot submit a test_run
without having submitted a test_build and an mpi_install result).  
The

bug caused the client and the server to become confused on what the
proper 'id' should be and when the submit script attempted to  
'guess'

the correct run it was unsuccessful and errored out.

So sorry this bug lived this long, but it should be fixed now.

-- Josh

On Aug 28, 2007, at 10:16 AM, Jeff Squyres wrote:


Josh found the problem and is in the process of fixing it.  DB
submits are currently disabled while Josh is working on the fix.
More specific details coming soon.

Unfortunately, it looks like all data from last night will be
junk.  :-(  You might as well kill any MTT scripts that are still
running from last night.


On Aug 28, 2007, at 9:14 AM, Jeff Squyres wrote:


Josh and I are investigating -- the total runs in the db in the
summary report from this morning is far too low.  :-(


On Aug 28, 2007, at 9:13 AM, Tim Prins wrote:


It installed and the tests built and made it into the database:
http://www.open-mpi.org/mtt/reporter.php?do_redir=293

Tim

Jeff Squyres wrote:

Did you get a correct MPI install section for mpich2?

On Aug 28, 2007, at 9:05 AM, Tim Prins wrote:


Hi all,

I am working with the jms branch, and when trying to use  
mpich2,

I get
the following submit error:

*** WARNING: MTTDatabase server notice:
mpi_install_section_name is
not in
 mtt database.
 MTTDatabase server notice: number_of_results is not in mtt
database.
 MTTDatabase server notice: phase is not in mtt database.
 MTTDatabase server notice: test_type is not in mtt
database.
 MTTDatabase server notice: test_build_section_name is
not in
mtt
 database.
 MTTDatabase server notice: variant is not in mtt database.
 MTTDatabase server notice: command is not in mtt database.
 MTTDatabase server notice: fields is not in mtt database.
 MTTDatabase server notice: resource_manager is not in mtt
database.

 MTT submission for test run
 MTTDatabase server notice: Invalid test_build_id (47368)
given.
 Guessing that it should be -1
 MTTDatabase server error: ERROR: Unable to find a
test_build to
 associate with this test_run.

 MTTDatabase abort: (Tried to send HTTP error) 400
 MTTDatabase abort:
 No test_build associated with this test_run
*** WARNING: MTTDatabase did not get a serial; phases will be
isolated from
 each other in the reports
Reported to MTTDatabase: 1 successful submit, 0 failed  
submits

(total of

   12 results)

This happens for each test run section.

Thanks,

Tim
___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users






___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users



--
Jeff Squyres
Cisco Systems

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users



--
Jeff Squyres
Cisco Systems

___
mtt-users mailing 

Re: [MTT devel] [MTT users] Database submit error

[Jeff Squyres]

No objections.  If the data is junk, just ditch it.

On Aug 31, 2007, at 12:47 PM, Josh Hursey wrote:


I was looking at the data from Monday Aug 27, 8 am to Tuesday Aug 28,
Noonish when this problem was occuring, and the data is mostly
invalid. We have test_builds pointing at the wrong test_suites. Since
this brings all of this data inso suspicion I'm going through and
flaging them all as 'trial'.

If you don't have any conflict, then I'd like to remove this data
alltogether from the database so the normalization tables can be
cleaned up.

Any objections to removing the set of data in the time range Monday
Aug 27, 8 am to Tuesday Aug 28, Noonish? it's about 8,000 test_runs
since most of the test runs were getting rejected during that time
period we are not losing any good data.

-- Josh


On Aug 28, 2007, at 10:27 AM, Josh Hursey wrote:


Short Version:
--
I just finished the fix, and the submit script is back up and  
running.


This was a bug that arose in testing, but somehow did not get
propagated to the production database.

Long Version:
-
The new databases uses partition tables to archive test results. As
part of this there are some complex rules to mask the partition table
complexity from the users of the db. There was a bug in the insert
rule in which the 'id' of the submitted result (mpi_install,
test_build, and test_run) was a different value than expected since
the 'id' was not translated properly to the partition table setup.

The fix was to drop all rules and replace them with the correct
versions. The submit errors you saw below were caused by integrity
checks in the submit script that keep data from being submitted that
do not have a proper lineage (e.g., you cannot submit a test_run
without having submitted a test_build and an mpi_install result). The
bug caused the client and the server to become confused on what the
proper 'id' should be and when the submit script attempted to 'guess'
the correct run it was unsuccessful and errored out.

So sorry this bug lived this long, but it should be fixed now.

-- Josh

On Aug 28, 2007, at 10:16 AM, Jeff Squyres wrote:


Josh found the problem and is in the process of fixing it.  DB
submits are currently disabled while Josh is working on the fix.
More specific details coming soon.

Unfortunately, it looks like all data from last night will be
junk.  :-(  You might as well kill any MTT scripts that are still
running from last night.


On Aug 28, 2007, at 9:14 AM, Jeff Squyres wrote:


Josh and I are investigating -- the total runs in the db in the
summary report from this morning is far too low.  :-(


On Aug 28, 2007, at 9:13 AM, Tim Prins wrote:


It installed and the tests built and made it into the database:
http://www.open-mpi.org/mtt/reporter.php?do_redir=293

Tim

Jeff Squyres wrote:

Did you get a correct MPI install section for mpich2?

On Aug 28, 2007, at 9:05 AM, Tim Prins wrote:


Hi all,

I am working with the jms branch, and when trying to use mpich2,
I get
the following submit error:

*** WARNING: MTTDatabase server notice:
mpi_install_section_name is
not in
 mtt database.
 MTTDatabase server notice: number_of_results is not in mtt
database.
 MTTDatabase server notice: phase is not in mtt database.
 MTTDatabase server notice: test_type is not in mtt  
database.
 MTTDatabase server notice: test_build_section_name is  
not in

mtt
 database.
 MTTDatabase server notice: variant is not in mtt database.
 MTTDatabase server notice: command is not in mtt database.
 MTTDatabase server notice: fields is not in mtt database.
 MTTDatabase server notice: resource_manager is not in mtt
database.

 MTT submission for test run
 MTTDatabase server notice: Invalid test_build_id (47368)
given.
 Guessing that it should be -1
 MTTDatabase server error: ERROR: Unable to find a
test_build to
 associate with this test_run.

 MTTDatabase abort: (Tried to send HTTP error) 400
 MTTDatabase abort:
 No test_build associated with this test_run
*** WARNING: MTTDatabase did not get a serial; phases will be
isolated from
 each other in the reports

Reported to MTTDatabase: 1 successful submit, 0 failed submits
(total of

   12 results)

This happens for each test run section.

Thanks,

Tim
___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users






___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users



--
Jeff Squyres
Cisco Systems

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users



--
Jeff Squyres
Cisco Systems

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

Re: [MTT devel] [MTT svn] svn:mtt-svn r964

[Pak Lui]

Oh you mean why would anyone want to submit their intel test results to 
the database??


So the scenario is like this, I have a dev ws setup for a pending CMR 
for my PML changes. The changes are also in my tmp branch. The point of 
running the test to make sure my new dev stuff works before putting back 
to v1.2 branch.


So I guess one could argue I could have MTT to bring over the test from 
tmp branch instead of my dev ws to run the tests.


Ethan Mallove wrote:

Pak, we're scratching our heads on this new "developer
results into central database" use case. Could you give more
details?

-Ethan


On Thu, Aug/30/2007 01:32:26PM, Jeff Squyres wrote:

Ethan --

Did Pak really mean to submit to the database?

On Aug 30, 2007, at 1:22 PM, jjhur...@osl.iu.edu wrote:


Author: jjhursey
Date: 2007-08-30 13:22:06 EDT (Thu, 30 Aug 2007)
New Revision: 964
URL: https://svn.open-mpi.org/trac/mtt/changeset/964

Log:
Increase the 'mpi_version' char length due to the following error:

SQL QUERY: INSERT INTO mpi_get
(mpi_get_id, mpi_name, mpi_version) VALUES
('507', 'developer',
'_workspace_paklui_ompi-paklui-1.2-pml_shared- 
install_SunOS_sparc_2007.08.29_bin_mpirun')

SQL ERROR: ERROR:  value too long for type character varying(32)
SQL ERROR:



Text files modified:
   trunk/server/sql/v3/schemas-v3.sql | 2 +-
   1 files changed, 1 insertions(+), 1 deletions(-)

Modified: trunk/server/sql/v3/schemas-v3.sql
== 


--- trunk/server/sql/v3/schemas-v3.sql  (original)
+++ trunk/server/sql/v3/schemas-v3.sql	2007-08-30 13:22:06 EDT  
(Thu, 30 Aug 2007)

@@ -107,7 +107,7 @@
 -- Current Max: 21 chars
 mpi_namevarchar(64) NOT NULL DEFAULT 'bogus',
 -- Current Max: 24 chars
-mpi_version varchar(32) NOT NULL DEFAULT 'bogus',
+mpi_version varchar(128) NOT NULL DEFAULT 'bogus',

 UNIQUE (
 mpi_name,
___
mtt-svn mailing list
mtt-...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--

- Pak Lui
pak@sun.com

Re: [MTT devel] [MTT svn] svn:mtt-svn r964

[Ethan Mallove]

Pak, we're scratching our heads on this new "developer
results into central database" use case. Could you give more
details?

-Ethan


On Thu, Aug/30/2007 01:32:26PM, Jeff Squyres wrote:
> Ethan --
> 
> Did Pak really mean to submit to the database?
> 
> On Aug 30, 2007, at 1:22 PM, jjhur...@osl.iu.edu wrote:
> 
> > Author: jjhursey
> > Date: 2007-08-30 13:22:06 EDT (Thu, 30 Aug 2007)
> > New Revision: 964
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/964
> >
> > Log:
> > Increase the 'mpi_version' char length due to the following error:
> >
> > SQL QUERY: INSERT INTO mpi_get
> > (mpi_get_id, mpi_name, mpi_version) VALUES
> > ('507', 'developer',
> > '_workspace_paklui_ompi-paklui-1.2-pml_shared- 
> > install_SunOS_sparc_2007.08.29_bin_mpirun')
> > SQL ERROR: ERROR:  value too long for type character varying(32)
> > SQL ERROR:
> >
> >
> >
> > Text files modified:
> >trunk/server/sql/v3/schemas-v3.sql | 2 +-
> >1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/server/sql/v3/schemas-v3.sql
> > == 
> > 
> > --- trunk/server/sql/v3/schemas-v3.sql  (original)
> > +++ trunk/server/sql/v3/schemas-v3.sql  2007-08-30 13:22:06 EDT  
> > (Thu, 30 Aug 2007)
> > @@ -107,7 +107,7 @@
> >  -- Current Max: 21 chars
> >  mpi_namevarchar(64) NOT NULL DEFAULT 'bogus',
> >  -- Current Max: 24 chars
> > -mpi_version varchar(32) NOT NULL DEFAULT 'bogus',
> > +mpi_version varchar(128) NOT NULL DEFAULT 'bogus',
> >
> >  UNIQUE (
> >  mpi_name,
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**

[Jeff Squyres]

I think probably the easiest thing to do is just mv reporter.php and  
submit.php so that mtt clients and individuals browsing will not be  
able to access it.



On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote:


Jeff and Ethan,

How do I shutdown the MTT server? Do I just log into milliways as
mtt, or can I disable it from my user?

Cheers,
-- Josh

Begin forwarded message:


From: Josh Hursey 
Date: August 24, 2007 1:37:18 PM EDT
To: General user list for the MPI Testing Tool 
Subject: [MTT users] MTT Database and Reporter Upgrade **Action
Required**
Reply-To: General user list for the MPI Testing Tool 

Short Version:
--
The MTT development group is rolling out newly optimized web frontend
and backend database. As a result we will be taking down the MTT site
at IU Monday, August 27 from 8 am to Noon US eastern time.

During this time you will not be able to submit data to the MTT
database. Therefore you need to disable any runs that will report
during this time or your client will fail with unable to connect to
server messages.

This change does not affect the client configurations, so MTT users
do *not* need to update their clients at this time.


Longer Version:
---
The MTT development team has been working diligently on server side
optimizations over the past few months. This work involved major
changes to the database schema, web reporter, and web submit
components of the server.

We want to roll out the new server side optimizations on Monday, Aug.
27. Given the extensive nature of the improvements the MTT server
will need to be taken down for a few hours for this upgrade to take
place. We are planning on taking down the MTT server at 8 am and
we hope to have it back by Noon US Eastern time.

MTT users that would normally submit results during this time range
will need to disable their runs, or they will see server error
messages during this outage.

This upgrade does not require any client changes, so outside of the
down time contributors need not change or upgrade their MTT
installations.

Below are a few rough performance numbers illustrating the difference
between the old and new server versions as seen by the reporter.

Summary report: 24 hours, all orgs
 87 sec - old version
  6 sec - new version
Summary report: 24 hours, org = 'iu'
 37 sec - old
  4 sec - new
Summary report: Past 3 days, all orgs
138 sec - old
  9 sec - new
Summary report: Past 3 days, org = 'iu'
 49 sec - old
 11 sec - new
Summary report: Past 2 weeks, all orgs
863 sec - old
 34 sec - new
Summary report: Past 2 weeks, org = 'iu'
878 sec - old
 12 sec - new
Summary report: Past 1 month, all org
   1395 sec - old
158 sec - new
Summary report: Past 1 month, org = 'iu'
   1069 sec - old
 39 sec - new
Summary report: (2007-06-18 - 2007-06-19), all org
484 sec - old
  5 sec - new
Summary report: (2007-06-18 - 2007-06-19), org = 'iu'
479 sec - old
  2 sec - new

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**

[Josh Hursey]

Jeff and Ethan,

How do I shutdown the MTT server? Do I just log into milliways as  
mtt, or can I disable it from my user?


Cheers,
-- Josh

Begin forwarded message:


From: Josh Hursey 
Date: August 24, 2007 1:37:18 PM EDT
To: General user list for the MPI Testing Tool 
Subject: [MTT users] MTT Database and Reporter Upgrade **Action  
Required**
Reply-To: General user list for the MPI Testing Tool us...@open-mpi.org>


Short Version:
--
The MTT development group is rolling out newly optimized web frontend
and backend database. As a result we will be taking down the MTT site
at IU Monday, August 27 from 8 am to Noon US eastern time.

During this time you will not be able to submit data to the MTT
database. Therefore you need to disable any runs that will report
during this time or your client will fail with unable to connect to
server messages.

This change does not affect the client configurations, so MTT users
do *not* need to update their clients at this time.


Longer Version:
---
The MTT development team has been working diligently on server side
optimizations over the past few months. This work involved major
changes to the database schema, web reporter, and web submit
components of the server.

We want to roll out the new server side optimizations on Monday, Aug.
27. Given the extensive nature of the improvements the MTT server
will need to be taken down for a few hours for this upgrade to take
place. We are planning on taking down the MTT server at 8 am and
we hope to have it back by Noon US Eastern time.

MTT users that would normally submit results during this time range
will need to disable their runs, or they will see server error
messages during this outage.

This upgrade does not require any client changes, so outside of the
down time contributors need not change or upgrade their MTT
installations.

Below are a few rough performance numbers illustrating the difference
between the old and new server versions as seen by the reporter.

Summary report: 24 hours, all orgs
 87 sec - old version
  6 sec - new version
Summary report: 24 hours, org = 'iu'
 37 sec - old
  4 sec - new
Summary report: Past 3 days, all orgs
138 sec - old
  9 sec - new
Summary report: Past 3 days, org = 'iu'
 49 sec - old
 11 sec - new
Summary report: Past 2 weeks, all orgs
863 sec - old
 34 sec - new
Summary report: Past 2 weeks, org = 'iu'
878 sec - old
 12 sec - new
Summary report: Past 1 month, all org
   1395 sec - old
158 sec - new
Summary report: Past 1 month, org = 'iu'
   1069 sec - old
 39 sec - new
Summary report: (2007-06-18 - 2007-06-19), all org
484 sec - old
  5 sec - new
Summary report: (2007-06-18 - 2007-06-19), org = 'iu'
479 sec - old
  2 sec - new

___
mtt-users mailing list
mtt-us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users