[MTT devel] Fwd: [MTT commits] Git: open-mpi/mtt branch gh-pages updated. a919925665b72524e7c77358c94c1d677900156b

[Jeff Squyres (jsquyres) via mtt-devel]

Is there a way, perchance, to not re-generate / re-push the docs if nothing 
meaningful changes?

E.g., this commit is just changing the date, but no other content changed.



Begin forwarded message:

From: gitdub-nore...@aws.open-mpi.org<mailto:gitdub-nore...@aws.open-mpi.org> 
('Gitdub ')
Subject: [MTT commits] Git: open-mpi/mtt branch gh-pages updated. 
a919925665b72524e7c77358c94c1d677900156b
Date: August 16, 2018 at 2:36:43 PM EDT
To: mtt-comm...@lists.open-mpi.org<mailto:mtt-comm...@lists.open-mpi.org>
Reply-To: mailto:mtt-de...@open-mpi.org>>

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "open-mpi/mtt".

The branch, gh-pages has been updated
  via  a919925665b72524e7c77358c94c1d677900156b (commit)
 from  3de9e99a5e7f2a5b1f21fa6253d3877bf2bf1621 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
https://github.com/open-mpi/mtt/commit/a919925665b72524e7c77358c94c1d677900156b

commit a919925665b72524e7c77358c94c1d677900156b
Author: Deployment Bot (from Travis CI) 
Date:   Thu Aug 16 18:36:39 2018 +

   Deploy open-mpi/mtt to github.com/open-mpi/mtt.git:gh-pages

diff --git a/latex/refman.tex b/latex/refman.tex
index 7d49b40..dbfac3c 100644
--- a/latex/refman.tex
+++ b/latex/refman.tex
@@ -67,8 +67,8 @@
\fancyhead[RO]{\fancyplain{}{\bfseries\thepage}}
\fancyfoot[LE]{\fancyplain{}{}}
\fancyfoot[CE]{\fancyplain{}{}}
-\fancyfoot[RE]{\fancyplain{}{\bfseries\scriptsize Generated on Thu Aug 16 2018 
18\-:36\-:21 for pymtt by Doxygen }}
-\fancyfoot[LO]{\fancyplain{}{\bfseries\scriptsize Generated on Thu Aug 16 2018 
18\-:36\-:21 for pymtt by Doxygen }}
+\fancyfoot[RE]{\fancyplain{}{\bfseries\scriptsize Generated on Thu Aug 16 2018 
18\-:36\-:28 for pymtt by Doxygen }}
+\fancyfoot[LO]{\fancyplain{}{\bfseries\scriptsize Generated on Thu Aug 16 2018 
18\-:36\-:28 for pymtt by Doxygen }}
\fancyfoot[CO]{\fancyplain{}{}}
\fancyfoot[RO]{\fancyplain{}{}}
\renewcommand{\footrulewidth}{0.4pt}
@@ -120,7 +120,7 @@
\vspace*{1cm}
{\large Generated by Doxygen 1.8.6}\\
\vspace*{0.5cm}
-{\small Thu Aug 16 2018 18:36:21}\\
+{\small Thu Aug 16 2018 18:36:28}\\
\end{center}
\end{titlepage}
\clearemptydoublepage


---

Summary of changes:
latex/refman.tex | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)


hooks/post-receive
--
open-mpi/mtt
___
mtt-commits mailing list
mtt-comm...@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-commits


--
Jeff Squyres
jsquy...@cisco.com<mailto:jsquy...@cisco.com>

___
mtt-devel mailing list
mtt-devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/mtt-devel

[MTT devel] This list is migrating!

[Jeff Squyres (jsquyres)]

Short version
=

The server for this mailing list will be migrating sometime soon (the exact 
timing is not fully predictable).  Three things you need to know:

1. We'll send a "This list is now closed for migration" last message when the 
migration starts
2. We'll send a "This list is now open again" first message when the migration 
completes
3. The list email address will move from @open-mpi.org to @lists.open-mpi.org

More detail
===

The Open MPI hosting infrastructure is slowly moving away from its home of 10+ 
years: our gracious hosts at Indiana University (thank you for all the help and 
support, IU!).  The next pieces to migrate are the Open MPI project mailing 
lists (including this one).

The exact timing of the migration depends on our new hosting provider vendor; 
it's quite difficult to give an exact timeline.  The procedure will generally 
be something like this:

1. Send the final "This list is now closed!" email across this list
2. Shut off all incoming mail to the list
3. Shut down the web pages that allow users to make changes to the list
4. Bundle up all the list data and send it to our new hosting provider
5. Work with the provider to get the new lists online
6. Send a "This list is now open again!" email across the list

As noted above, we're changing the hostnames on the mailing lists to 
@lists.open-mpi.org so that we can de-couple the mailing lists from the rest of 
the web hosting infrastructure.  Please update your addressbook and mail 
filters appropriately.

Webified archives of the mailing lists will continue to be available:

1. Once the migration completes, the existing web archives (under, for example, 
https://www.open-mpi.org/community/lists/users/) will continue to be available, 
but they'll be frozen -- no new messages will be added there.  Specifically: 
links to old posts will continue to work.
2. New web archives for all the lists -- to include all the old posts -- will 
become available elsewhere.  Specifics will be included in the "The list is now 
open again!" mail.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] MTT ompi-tests password

[Jeff Squyres (jsquyres)]

Folks --

In helping someone get up to speed with MTT yesterday, I updated an MTT sample 
.ini file and accidentally committed the ompiteam-mtt Github password to the 
public repository.  :-(

I have therefore just changed the password for the ompiteam-mtt account.  
Please contact me off-list for the updated password.

Sorry for the inconvenience.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] GitHub Issue Cleanup

[Jeff Squyres (jsquyres)]

+1

> On May 13, 2016, at 2:27 PM, Ralph Castain <r...@open-mpi.org> wrote:
> 
> +1 on all these
> 
> Thanks for taking point, Josh!
> 
>> On May 13, 2016, at 11:08 AM, Josh Hursey <jjhur...@open-mpi.org> wrote:
>> 
>> We are seeing more activity with MTT development, and there is a desire to 
>> push to a formal release at some point in the not-to-distant future. As 
>> such, I think it is time to clean up/out the issues on GitHub. Quite a 
>> number of those issues are feature ideas that we came up with that were 
>> never investigated.
>> 
>> So I propose that we do the following. All of which is open for discussion, 
>> and I can take point on making the changes once we settle on what we want.
>> 
>> Milestones:
>>  1) Mark all existing milestones as completed.
>>  2) Create a v4.0 milestone to track development to the 'first' release (why 
>> not v1.0 - see [A] below)
>>  3) Any issue filed against "Future" will be filed instead against 
>> "ArchivedFuture"
>> 
>> Labels:
>>  1) Create a "work in progress" label - for PRs in progress
>>  2) Create a label for each of the major parts of MTT
>> - "Perl Client"
>> - "Python Client"
>> - "Reporter"
>> - "Database"
>> - "Server"
>>  3) Create a "Wishlist" label where we can label wild enhancement ideas that 
>> we would like, but know we have no time to pursue in the near future. That 
>> way it is easy to get a list of neat things to do for people wanting to jump 
>> in.
>>  4) Create an "Archive" label
>> 
>> Issues:
>>  1) All existing issues get labeled with "Archive" in addition to their 
>> existing labels
>> 
>> 
>> What do folks think? Did I miss anything?
>> 
>> Thanks,
>> Josh
>> 
>> 
>> [A] There was informal v1.0 / v2.0 / v3.0 waypoints in the history. I didn't 
>> want to suggest removing those incase that history is important to us in the 
>> future. However, I'm open to discussing removing them too.
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>> Searchable archives: 
>> http://www.open-mpi.org/community/lists/mtt-devel/2016/05/0657.php
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/mtt-devel/2016/05/0657.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [OMPI devel] mtt-submit, etc.

[Jeff Squyres (jsquyres)]

I see the issue in the current code:

1. The current code assumes that if you use the MTT database reporter, you can 
reach the database.  One of the first things it does is ping the server to 
ensure that it's reachable.  The rationale is that you don't want MTT to run 
for a long time and then find out that you can't reach the database to submit 
your results.

2. mtt-relay can certainly be used, but not all sites permit using such a 
thing.  Ditto with an ssh tunnel.

3. Looking at the code, it's probably possible to allow the INI file to specify 
no URL but specify a $debug_filename, and in such cases, ignore some of the 
database-submit code.  E.g.:
   - Skip this block 
https://github.com/open-mpi/mtt/blob/master/lib/MTT/Reporter/MTTDatabase.pm#L178-L203
   - Skip this block 
https://github.com/open-mpi/mtt/blob/master/lib/MTT/Reporter/MTTDatabase.pm#L374-L419
   - Modify this conditional 
https://github.com/open-mpi/mtt/blob/master/lib/MTT/Reporter/MTTDatabase.pm#L424

I'm afraid I don't have the cycles to look deeper into this at the moment, but 
this is where I would start.



> On Oct 23, 2015, at 12:20 AM, George Bosilca <bosi...@icl.utk.edu> wrote:
> 
> If you are able to connect (assuming ssh) to the nodes that will execute the 
> tests, why can’t you simply use an ssh tunnel to contact the IU server ?
> 
>   George.
> 
>> On Oct 23, 2015, at 00:08 , Ralph Castain <r...@open-mpi.org> wrote:
>> 
>> I was thinking about this, and I believe it would require a change to the 
>> mtt client to avoid it. I’m working on a new Python-based version of it, and 
>> I’ll make sure to deal with this there.
>> 
>> In the interim, I’ll have to defer to some old, gray Perl guru to update the 
>> current client
>> 
>> 
>>> On Oct 22, 2015, at 9:23 AM, Howard Pritchard <hpprit...@gmail.com> wrote:
>>> 
>>> Hi Folks,
>>> 
>>> I don't seem to have gotten subscribed yet to mtt-users mail list so
>>> forwarding to the dev team.
>>> 
>>> Howard
>>> 
>>> -- Forwarded message --
>>> From: Howard Pritchard <hpprit...@gmail.com>
>>> Date: 2015-10-22 10:18 GMT-06:00
>>> Subject: mtt-submit, etc.
>>> To: mtt-us...@open-mpi.org
>>> 
>>> 
>>> HI Folks,
>>> 
>>> I have the following issue with a cluster I would like to use for 
>>> submitting MTT results
>>> for Open MPI, namely, that the nodes on which I have to submit batch jobs 
>>> to run
>>> the tests don't have external internet connectivity, so if my mtt ini file 
>>> has a IU database reporter
>>> section, the run dies in the "ping the mtt server" test.
>>> 
>>> What I have right now is a two-stage process where I checkout and 
>>> compile/build
>>> Open MPI and the tests on a front end which does have access to the mtt 
>>> server.
>>> This part works and gets reported back to IU database. 
>>> 
>>> I can run the tests using mtt, but have to disable all the mtt server 
>>> reporter stuff.
>>> 
>>> I thought I could use mtt-submit to submit some kind of mttdatabase debug 
>>> file
>>> back to IU once the batch job has completed, but I can't figure out a way
>>> to generate this file without enable the mtt server reporter section in the 
>>> ini file,
>>> and so back to the ping failure issue.
>>> 
>>> Would anyone have suggestions on how to work around this problem?
>>> 
>>> Thanks,
>>> 
>>> Howard
>>> 
>>> 
>>> ___
>>> devel mailing list
>>> de...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>> Link to this post: 
>>> http://www.open-mpi.org/community/lists/devel/2015/10/18244.php
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/devel/2015/10/18249.php
> 
> ___
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2015/10/18251.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] MTT: let's use git tags

[Jeff Squyres (jsquyres)]

The topic came up today that MTT sometimes has bugs, particularly w.r.t. 
ongoing MTT development.

It seems like we should use git tags to let the OMPI/testing community know 
which tag they should be using (vs. HEAD).

To that end, I have created a "v3.0.0" tag that exists before the controversial 
set of commits I pushed the other day -- e12386e.  Assumedly, when we fix 
whatever problem Mellanox is setting with commits beyond e12386e, we can call 
that "v3.0.1", or some such, and ask everyone to move up to it.

So those who need stability should stick back at tags, and those who want to 
help with development can be at the HEAD.

How does that sound?

If that sounds ok, I'll ask the OMPI test community to git checkout v3.0.0.  
And in the future, we'll ask the OMPI test community to update to the next 
relevant tag, etc.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

Ok, just got in to Chicago from my flight and am back online.

Mike: you are still not providing very much information.  :-\

Your first mails make it seem like MTT is continuing to run, but leaving 
"launchers" (assumedly mpirun processes) still running, but they have no 
children.  Which would be very weird for mpirun to do, if it has no children 
left.  This could be both an MTT and an ORTE bug, in this case.

But your last mail seems to imply that MTT is hanging indefinitely.

Can you please provide a clear, precise description of what is happening?

FWIW: Yes, we are killing the parent first now, to give mpirun a chance to 
cleanup / tell remote orteds to die / kill children processes / etc.  Killing 
the children first both doesn't test the common case of how people kill MPI 
processes (i.e., they kill mpirun), and it also doesn't allow mpirun to tell 
remote processes to die.

Do you run with --verbose output?  MTT should output messages like "*** Killing 
mpirun with SIGTERM", and the like.  Do you see timeout messages at all?  I.e., 
is MTT not entering the timeout code at all?

...etc.



On Jun 23, 2014, at 12:16 PM, Dave Goodell (dgoodell) <dgood...@cisco.com> 
wrote:

> On Jun 23, 2014, at 8:48 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:
> 
>> btw, i think now, when parent process is killed before child, OS makes child 
>> as "" which stick around for good.
> 
> The grandparent should inherit the child.  If the grandparent then does not 
> wait(2) on the child, then the child will remain a zombie / defunct.  So in 
> our specific case, this behavior will depend on what the parent process of 
> mpirun is and whether it is waiting on child processes appropriately.
> 
> -Dave
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> Link to this post: 
> http://www.open-mpi.org/community/lists/mtt-devel/2014/06/0633.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

On Jun 23, 2014, at 7:47 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> after patch, it killed child processes but kept mpirun ... itself.

What does that mean -- are you saying that mpirun is still running?  Was mpirun 
sent a signal at all?  What kind of messages are being displayed?  ...etc.

The commits fix important bugs for me and others.  Clearly, there's still 
something not right.  And of course I'm willing to track it down.  But I can't 
help you if you just say "it doesn't work."

> before that patch - all processes were killed (and you are right, "mpirun 
> died right at the end of the timeout" was reported)

...which led to many months of misleading ORTE debugging, BTW.  :-\  That's why 
this commit was introduced into MTT -- in the quest of finally fixing both the 
mysterious ORTE hangs and the erroneous timeouts/"mpirun died right at the end" 
messages.

> but at least it left the cluster in the clean state w/o leftovers.
> now many "orphan" launchers  are alive from previous invocations.

Does "launchers" = mpirun?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] GIT: MTT branch master updated. 016088f2a0831b32ab5fd6f60f4cabe67e92e594

[Jeff Squyres (jsquyres)]

There was actually quite a bit of testing before this was committed.  This 
commit resolved a lot of hangs across multiple organizations.

Can you be more specific as to what is happening?

The prior code was killing child processes before mpirun itself, for example, 
which has led MTT to erroneously report that mpirun died right at the end of 
the timeout without being killed.  This has been ongoing for many months, at a 
minimum.




On Jun 23, 2014, at 4:37 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> this commit does more harm then good.
> we experience following:
> 
> - some child processes still running after timeout and mtt killed the job.
> 
> before this commit - it worked fine.
> please revert and test more.
>  
> 
> 
> On Sat, Jun 21, 2014 at 3:30 PM, MPI Team <mpit...@crest.iu.edu> wrote:
> The branch, master has been updated
>via  016088f2a0831b32ab5fd6f60f4cabe67e92e594 (commit)
>via  7fb4c6a4c9d71be127ea53bd463178510577f71f (commit)
>via  381ba177d835a54c3197d846f5a4edfc314efe27 (commit)
>via  cfdd29de2012eeb7592706f00dd07a52dd48cf6b (commit)
>via  940030ca20eb1eaf256e898b83866c1cb83aca5c (commit)
>   from  c99ed7c7b159a2cab58a251bd7c0dad8972ff901 (commit)
> 
> Those revisions listed above that are new to this repository have
> not appeared on any other notification email; so we list those
> revisions in full, below.
> 
> - Log -
> https://github.com/open-mpi/mtt/commit/016088f2a0831b32ab5fd6f60f4cabe67e92e594
> 
> commit 016088f2a0831b32ab5fd6f60f4cabe67e92e594
> Author: Jeff Squyres <jsquy...@cisco.com>
> Date:   Sat Jun 21 04:58:45 2014 -0700
> 
> DoCommand: several fixes to kill_proc logic
> 
> 1. Fix the kill(0, $pid) test to see if the process was still alive.
> 
> 2. Rename _kill_proc() to _kill_proc_tree() to indicate that it's
> really killing not only the PID in question, but also all of its
> descendants.
> 
> 3. In _kill_proc_tree(), change the order to kill the main PID first,
> and ''then'' kill all the descendants.
> 
> The main use case is when killing mpirun: if we kill mpirun's
> descendants ''first'', mpirun will detect its childrens' deaths and
> then cleanup and exit.  Later, when MTT finally gets around to killing
> mpirun, MTT will detect that mpirun is already dead and therefore emit
> a confusing "mpirun died right at end of timeout" message.  This is
> misleading at best; it doesn't indicate what actually happened.
> 
> However, if we kill mpirun first, it will take care of killing all of
> its descendants.  MTT will therefore emit the right messages about
> killing mpirun.  MTT will then redundantly try to kill a bunch of
> now-nonexistent descendant processes of mpirun, but that's ok/safe.
> We actually ''want'' this try-to-kill-mpirun's-descendants behavior to
> handle the case when mpirun is misbehaving / not cleaning up its
> descendants.
> 
> 4. DoCommand() is used for more than launching mpirun, so pass down
> $argv0 so that we can print the actual command name that is being
> killed in various Verbose/Debug messages, not the hard-coded "mpirun"
> string (which, in practice, was probably almost always correct, but
> still...).
> ---
>  lib/MTT/DoCommand.pm | 78 
> 
>  1 file changed, 55 insertions(+), 23 deletions(-)
> 
> diff --git a/lib/MTT/DoCommand.pm b/lib/MTT/DoCommand.pm
> index 02cdb94..646ca31 100644
> --- a/lib/MTT/DoCommand.pm
> +++ b/lib/MTT/DoCommand.pm
> @@ -2,7 +2,7 @@
>  #
>  # Copyright (c) 2005-2006 The Trustees of Indiana University.
>  # All rights reserved.
> -# Copyright (c) 2006-2013 Cisco Systems, Inc.  All rights reserved.
> +# Copyright (c) 2006-2014 Cisco Systems, Inc.  All rights reserved.
>  # Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights reserved.
>  # Copyright (c) 2007-2012 High Performance Computing Center Stuttgart,
>  # University of Stuttgart.  All rights reserved.
> @@ -57,23 +57,27 @@ sub DoCommand {
>  ($time_arg, $no_execute) = @_;
>  }
> 
> +# This function is called for killing both mpirun and each of its
> +# descendants.  We really only want to see verbose output for when we
> +# kill mpirun itself, so only show output when the caller provides us
> +# with a $argv0 value.
>  sub _kill_proc_one {
> -my ($pid) = @_;
> +my ($pid, $argv0) = @_;
> 
>  # How long to wait after each kill()
>  my $wait_time = 5;
> 
>  # See if the proc is alive first
> 

Re: [MTT devel] Converted to git

[Jeff Squyres (jsquyres)]

I assume this means that no one found any problems or has any changes.

I'll be moving this to its permanent home on github sometime soon and making 
the MTT SVN be read-only.  Trac will be migrating to be git-based soon as well.

Please do not use the MTT trac until further notice.  Thanks!


On Apr 16, 2014, at 3:40 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> wrote:

> BTW, I used the following SVN<-->email addresses mapping for creating the git 
> commits.  Let me know if you want something different:
> 
> adkulkar = Abhishek Kulkarni <adkul...@cs.indiana.edu>
> afriedle = Andrew Friedley <afrie...@osl.iu.edu>
> brbarret = Brian Barrett <brbar...@open-mpi.org>
> cyeoh = Chris Yeoh <cy...@au1.ibm.com>
> em162155 = Ethan Mallove <ethan.mall...@oracle.com>
> emallove = Ethan Mallove <ethan.mall...@oracle.com>
> eugene = Eugene Loh <eugene@oracle.com>
> hpcstork = Sven Stork <sven.st...@open-mpi.org>
> jjhursey = Josh Hursey <jjhur...@open-mpi.org>
> jsquyres = Jeff Squyres <jsquy...@cisco.com>
> miked = Mike Dubman <mi...@mellanox.com>
> pasha = Pavel Shamis <sham...@ornl.gov>
> rusraink = Rainer Keller <rainer.kel...@hft-stuttgart.de>
> shiqing = Shiqing Fan <f...@hlrs.de>
> timattox = Tim Mattox <timat...@open-mpi.org>
> 
> 
> On Apr 16, 2014, at 3:37 PM, "Jeff Squyres (jsquyres)" <jsquy...@cisco.com> 
> wrote:
> 
>> I have done a TRIAL conversion to git and pushed it to a demo repo at 
>> github.  Please examine it and let me know if you see any problems:
>> 
>>   https://github.com/jsquyres/mtt-test
>> 
>> Note that we converted references to "rXYZ" in log messages -- see 
>> https://github.com/jsquyres/mtt-test/commit/ebb98c67677b02fa00064f8b1ae0d40941c305cd
>>  for an example.
>> 
>> -- 
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to: 
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>> 
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] Converted to git

[Jeff Squyres (jsquyres)]

BTW, I used the following SVN<-->email addresses mapping for creating the git 
commits.  Let me know if you want something different:

adkulkar = Abhishek Kulkarni <adkul...@cs.indiana.edu>
afriedle = Andrew Friedley <afrie...@osl.iu.edu>
brbarret = Brian Barrett <brbar...@open-mpi.org>
cyeoh = Chris Yeoh <cy...@au1.ibm.com>
em162155 = Ethan Mallove <ethan.mall...@oracle.com>
emallove = Ethan Mallove <ethan.mall...@oracle.com>
eugene = Eugene Loh <eugene@oracle.com>
hpcstork = Sven Stork <sven.st...@open-mpi.org>
jjhursey = Josh Hursey <jjhur...@open-mpi.org>
jsquyres = Jeff Squyres <jsquy...@cisco.com>
miked = Mike Dubman <mi...@mellanox.com>
pasha = Pavel Shamis <sham...@ornl.gov>
rusraink = Rainer Keller <rainer.kel...@hft-stuttgart.de>
shiqing = Shiqing Fan <f...@hlrs.de>
timattox = Tim Mattox <timat...@open-mpi.org>


On Apr 16, 2014, at 3:37 PM, "Jeff Squyres (jsquyres)" <jsquy...@cisco.com> 
wrote:

> I have done a TRIAL conversion to git and pushed it to a demo repo at github. 
>  Please examine it and let me know if you see any problems:
> 
>https://github.com/jsquyres/mtt-test
> 
> Note that we converted references to "rXYZ" in log messages -- see 
> https://github.com/jsquyres/mtt-test/commit/ebb98c67677b02fa00064f8b1ae0d40941c305cd
>  for an example.
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] Converted to git

[Jeff Squyres (jsquyres)]

I have done a TRIAL conversion to git and pushed it to a demo repo at github.  
Please examine it and let me know if you see any problems:

https://github.com/jsquyres/mtt-test

Note that we converted references to "rXYZ" in log messages -- see 
https://github.com/jsquyres/mtt-test/commit/ebb98c67677b02fa00064f8b1ae0d40941c305cd
 for an example.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

On Apr 7, 2014, at 6:39 PM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> somehow we run it with both, --verbose not enough to understand the problem 
> and --debug is too much.
> 
> maybe --trace is here to rescue?

It won't really solve the problem; it'll just create another level of argument 
about where a given message should go (now there will be 3 levels instead of 
2... eventually there will be 4, and then 5, and then ...).

Do you really need to run with --debug all the time?  I.e., do you have so many 
problems with MTT itself that you need to run with --debug?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

Yes.

The intent is that --debug is *very* verbose, and is generally only useful 
when something goes wrong.

I run Cisco's automated MTT with only --verbose.



On Apr 7, 2014, at 6:35 PM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> ohh.. it is just flooding the log with same data for every test launch.
> 
> maybe we should have verbose level in mtt?
> 
> 
> On Mon, Apr 7, 2014 at 6:30 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> 
> wrote:
> Mike --
> 
> Why did you comment these out?  By definition, --debug output should be a LOT 
> of output.
> 
> 
> On Apr 5, 2014, at 7:27 PM, <svn-commit-mai...@open-mpi.org> wrote:
> 
> > Author: miked (Mike Dubman)
> > Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> > New Revision: 1637
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> >
> > Log:
> > silence print
> >
> > Text files modified:
> >   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++--
> >   1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> > ==
> > --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17 14:14:47 
> > 2014(r1636)
> > +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05 19:27:28 
> > EDT (Sat, 05 Apr 2014)  (r1637)
> > @@ -331,7 +331,7 @@
> >
> > # Check the environment for OMPI_MCA_* values
> > foreach my $e (keys(%ENV)) {
> > -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> > if ($e =~ m/^OMPI_MCA_(\S+)/) {
> > my $v = $ENV{"OMPI_MCA_$1"};
> > push(@params, "--env-mca $1 $v");
> > @@ -339,7 +339,7 @@
> > }
> >
> > $str = join(' ', @params);
> > -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> > $str;
> > }
> >
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1637 - trunk/lib/MTT/Values/Functions/MPI

[Jeff Squyres (jsquyres)]

Mike --

Why did you comment these out?  By definition, --debug output should be a LOT 
of output.


On Apr 5, 2014, at 7:27 PM, <svn-commit-mai...@open-mpi.org> wrote:

> Author: miked (Mike Dubman)
> Date: 2014-04-05 19:27:28 EDT (Sat, 05 Apr 2014)
> New Revision: 1637
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1637
> 
> Log:
> silence print
> 
> Text files modified: 
>   trunk/lib/MTT/Values/Functions/MPI/OMPI.pm | 4 ++-- 
>
>   1 files changed, 2 insertions(+), 2 deletions(-)
> 
> Modified: trunk/lib/MTT/Values/Functions/MPI/OMPI.pm
> ==
> --- trunk/lib/MTT/Values/Functions/MPI/OMPI.pmMon Mar 17 14:14:47 
> 2014(r1636)
> +++ trunk/lib/MTT/Values/Functions/MPI/OMPI.pm2014-04-05 19:27:28 EDT 
> (Sat, 05 Apr 2014)  (r1637)
> @@ -331,7 +331,7 @@
> 
> # Check the environment for OMPI_MCA_* values
> foreach my $e (keys(%ENV)) {
> -Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> +#Debug("Functions::MPI::OMPI: Checking env key: $e\n");
> if ($e =~ m/^OMPI_MCA_(\S+)/) {
> my $v = $ENV{"OMPI_MCA_$1"};
> push(@params, "--env-mca $1 $v");
> @@ -339,7 +339,7 @@
> }
> 
> $str = join(' ', @params);
> -Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> +#Debug("Functions::MPI::OMPI: Returning MCA params $str\n");
> $str;
> }
> 
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] Migrating www.open-mpi.org

[Jeff Squyres (jsquyres)]

All --

Our hosting provider will be migrating 
www.open-mpi.org<http://www.open-mpi.org> to a new machine on Wednesday.  See 
message below for details.


Begin forwarded message:

From: DongInn Kim <di...@cs.indiana.edu<mailto:di...@cs.indiana.edu>>
Subject: Migrating www.open-mpi.org<http://www.open-mpi.org> from milliways to 
lion
List-Post: mtt-devel@lists.open-mpi.org
Date: August 5, 2013 11:53:38 AM PDT

Dear Open MPI developers and users,

We are planning to move all the services under 
www.open-mpi.org<http://www.open-mpi.org/> to the new server on Wednesday, Aug 
7th, 2013.
This migration may need some outage time of web services (e.g., 
http://www.open-mpi.org<http://www.open-mpi.org/>) and mailing list services 
(e.g., us...@open-mpi.org<mailto:us...@open-mpi.org>, 
de...@open-mpi.org<mailto:de...@open-mpi.org>, …).

The migration schedule is following:
- Date: Wednesday, Aug 7th, 2013
- Time:
6:00am-8:00am Pacific US time
7:00am-9:00am Mountain US time
8:00am-10:00am Central US time
9:00am-11:00am Eastern US time
1:00pm-3:00pm GMT

The following services would not be available during the migration.

- Web services (e.g., www.open-mpi.org<http://www.open-mpi.org/>)
- mailing lists:
  ad...@open-mpi.org<mailto:ad...@open-mpi.org>
  announce
  bugs
  devel
  devel-core
  docs
  ft
  hwloc-announce
  hwloc-bugs
  hwloc-devel
  hwloc-svn
  hwloc-users
  llamas
  mtt-announce
  mtt-bugs
  mtt-devel
  mtt-devel-core
  mtt-results
  mtt-svn
  mtt-users
  ompi-user-docs-bugs
  ompi-user-docs-svn
  svn
  svn-docs
  svn-docs-full
  svn-full
  svn-private
  svn-private-full
  users
- Mail archives
  http://www.open-mpi.org/community/lists/
- Mercurial mirror
  Will disappear (it has long-since moved out to Bitbucket)

I hope that we will not lose any mails sent to the above mailing lists even 
during the migration but it would be really appreciated if you hold up sending 
emails and svn commit until the migration is done.

Please let me know if you have any questions or issues about this migration.

Regards,

--
- DongInn
---
CREST System administrator
Indiana University
Bloomington, IN


--
Jeff Squyres
jsquy...@cisco.com<mailto:jsquy...@cisco.com>
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] fix zombie commit

[Jeff Squyres (jsquyres)]

On Feb 26, 2013, at 2:11 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> On Mon, Feb 25, 2013 at 6:24 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> 
> wrote:
> >Looking at the code, you're checking for zombie status before MTT kills the 
> >proc.  Am I reading that right?
> I don`t think the order matters, if process is not Zombie yet and about to be 
> killed by MTT later - it is a good flow.
> If process is already Zombie - mtt will not be able to kill it anyway and and 
> can stop waiting and switch to the new task.

No, the _kill_proc() routine does both a kill() and a waitpid().  The waitpid() 
should reap the zombie.

I.e., if the process has died, MTT simply just hasn't reaped it yet.  Hence, 
it's a zombie.

> >If so, then it could well be that the process has exited but not yet been 
> >reaped (because _kill_proc() hasn't been invoked yet).  If this is the case, 
> >is the real cause of the problem that >the OUTread and ERRread aren't being 
> >closed when the child process exits, and therefore we keep looping looking 
> >for new output from them?
> yep, sounds like it can be the cause, need to look into this code.

Ok.  It would be interesting to see if the process dies, but:

1) MTT is still blocking in select() (i.e., OUTread and OUTerr aren't returning 
0 from sysread upon process death)

2) $done is somehow not getting set to 0, and therefore MTT is still looping 
until the timeout expires

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] fix zombie commit

[Jeff Squyres (jsquyres)]

On Feb 24, 2013, at 6:59 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote:

> What protection do you mean? Check that /proc/pid/status exists? It is done 
> in Grep()

Ah, excellent -- I hadn't noticed that.

> We observe that process which was launched by mtt and hangs (mtt detect 
> timeout and starts do_command procedure), later enters into "defunct" state.

Looking at the code, you're checking for zombie status before MTT kills the 
proc.  Am I reading that right?

If so, then it could well be that the process has exited but not yet been 
reaped (because _kill_proc() hasn't been invoked yet).  If this is the case, is 
the real cause of the problem that the OUTread and ERRread aren't being closed 
when the child process exits, and therefore we keep looping looking for new 
output from them?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] fix zombie commit

[Jeff Squyres (jsquyres)]

Mike --

Please protect this code better; MTT is also run on Solaris and OS X.

Also, can you describe more fully the case where zombies are being left behind 
by MTT?


On Feb 24, 2013, at 1:44 AM, <svn-commit-mai...@open-mpi.org> wrote:

> Author: miked (Mike Dubman)
> Date: 2013-02-24 01:44:31 EST (Sun, 24 Feb 2013)
> New Revision: 1589
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1589
> 
> Log:
> * fix: fork leaves zombie processes sometimes. temp fix: detect zombie and 
> proceed with tests.
> 
> Text files modified: 
>   trunk/lib/MTT/DoCommand.pm | 6 ++  
>   1 files changed, 6 insertions(+), 0 deletions(-)
> 
> Modified: trunk/lib/MTT/DoCommand.pm
> ==
> --- trunk/lib/MTT/DoCommand.pmWed Feb 20 12:41:12 2013(r1588)
> +++ trunk/lib/MTT/DoCommand.pm2013-02-24 01:44:31 EST (Sun, 24 Feb 
> 2013)  (r1589)
> @@ -641,6 +641,12 @@
> if (!$pid_exists) {
> Verbose("--> Process completed somehow at " . time() . ", 
> proceeding with tests\n");
> $resume_tests++;
> +} else {
> +my $matches = MTT::Files::Grep("zombie", "/proc/$pid/status");
> +if (@$matches) {
> +Verbose("--> Process become Zombie at " . time() . ", 
> proceeding with tests\n");
> +$resume_tests++;
> +}
> }
> # Remove the timeout sentinel file, if a timeout notify timeout value 
> is set
> if (defined($end_time) and time() > $end_time) {
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1575 - trunk/lib/MTT/Reporter

[Jeff Squyres (jsquyres)]

Wow!  Icky.  That should be reported upstream.


On Jan 15, 2013, at 1:28 PM, Mike Dubman <mi...@dev.mellanox.co.il>
 wrote:

> there is a die"" in the MongoDB.connect :(
> 
> On Mon, Jan 14, 2013 at 4:47 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> 
> wrote:
> That's weird.  Why would an "eval" fix this situation?
> 
> 
> On Jan 14, 2013, at 8:15 AM, svn-commit-mai...@open-mpi.org wrote:
> 
> > Author: miked (Mike Dubman)
> > Date: 2013-01-14 08:15:54 EST (Mon, 14 Jan 2013)
> > New Revision: 1575
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1575
> >
> > Log:
> > fixes #1696
> >
> > Text files modified:
> >   trunk/lib/MTT/Reporter/MTTMongodb.pm | 2 +-
> >   1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/lib/MTT/Reporter/MTTMongodb.pm
> > ==
> > --- trunk/lib/MTT/Reporter/MTTMongodb.pm  Fri Jan  4 14:39:57 2013  
> >   (r1574)
> > +++ trunk/lib/MTT/Reporter/MTTMongodb.pm  2013-01-14 08:15:54 EST (Mon, 
> > 14 Jan 2013)  (r1575)
> > @@ -117,7 +117,7 @@
> >
> >   if($enable_mongo == 1)
> >   {
> > - eval "$conn = MongoDB::Connection->new(host => $url);";
> > + $conn = MongoDB::Connection->new(host => $url);
> >   if(defined($conn))
> >       {
> >   $db = $conn->mlnx_mtt;
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> _______
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1481 - in trunk: client lib/MTT/Reporter

[Jeff Squyres]

Mike --

MongoDB is a NoSQL thingy, right?

Can you describe this plugin a bit?  Do you guys have some kind of reporter for 
MongoDB?


On Aug 1, 2012, at 5:46 AM,  wrote:

> Author: miked (Mike Dubman)
> Date: 2012-08-01 05:46:03 EDT (Wed, 01 Aug 2012)
> New Revision: 1481
> URL: https://svn.open-mpi.org/trac/mtt/changeset/1481
> 
> Log:
> add modified version mongobquery and MTTMongodb
> 
> Added:
>   trunk/client/mongobquery.pl   (contents, props changed)
>   trunk/lib/MTT/Reporter/MTTMongodb.pm
> 
> Added: trunk/client/mongobquery.pl
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ trunk/client/mongobquery.pl   2012-08-01 05:46:03 EDT (Wed, 01 Aug 
> 2012)  (r1481)
> @@ -0,0 +1,1018 @@
> +#!/usr/bin/perl
> +#
> +# Copyright (c) 2009
> +# $COPYRIGHT$
> +# 
> +# Additional copyrights may follow
> +# 
> +# $HEADER$
> +#
> +# Now that @INC is setup, bring in the modules
> +
> +#use strict;
> +#use warnings;
> +use LWP::UserAgent;
> +use HTTP::Request::Common;
> +use Data::Dumper;
> +use File::Basename;
> +use File::Temp;
> +use Config::IniFiles;
> +use YAML::XS;
> +use MongoDB;
> +use MongoDB::OID;
> +use YAML;
> +use YAML::Syck;
> +use DateTime;
> +
> +###
> +# Set variables
> +###
> +my $module_name=$0;
> +my $module_path=$0;
> +
> +$module_name=~s/([^\/\\]+)$//;
> +$module_name=$1;
> + 
> +$module_path=~s/([^\/\\]+)$//;
> +
> +
> +###
> +# Main block
> +###
> +use Getopt::Long qw(:config no_ignore_case);
> +
> +my $opt_help;
> +my $opt_server;
> +my $opt_username;
> +my $opt_password;
> +
> +my $opt_ping;
> +my $opt_upload;
> +my $opt_query;
> +my $opt_view;
> +my $opt_admin;
> +
> +my @opt_data;
> +my @opt_raw;
> +
> +my $opt_gqls;
> +my @opt_gqlf;
> +my @opt_section;
> +my $opt_dir;
> +my $opt_no_raw;
> +
> +my $opt_dstore;
> +my $opt_info;
> +my $opt_format;
> +my $opt_mailto;
> +my $opt_regression_from;
> +my $opt_regression_to;
> +my $opt_regression_step;
> +
> +my @opt_newuser;
> +
> +GetOptions ("help|h" => \$opt_help,
> +"server|a=s" => \$opt_server,
> +"username|u=s" => \$opt_username,
> +"password|p=s" => \$opt_password,
> +"ping" => \$opt_ping,
> +"upload" => \$opt_upload,
> +"query" => \$opt_query,
> +"view" => \$opt_view,
> +"admin" => \$opt_admin,
> +
> +"data|S=s" => \@opt_data,
> +"raw|R=s" => \@opt_raw, 
> +
> +"gqls|L=s" => \$opt_gqls,
> +"gqlf|F=s" => \@opt_gqlf,
> +"section|T=s" => \@opt_section,
> +"dir|O=s" => \$opt_dir,
> +"no-raw" => \$opt_no_raw,
> +
> +"dstore|D" => \$opt_dstore,
> +"info|I=s" => \$opt_info,
> +"format|V=s" => \$opt_format,
> +"email|e=s" => \$opt_mailto,
> +
> +"newuser=s{3,5}" => \@opt_newuser,
> +
> + "regression-from=s" => \$opt_regression_from,
> + "regression-to=s" => \$opt_regression_to,
> + "regression-step=s" => \$opt_regression_step
> +);
> +
> +
> +my $url = ();
> +my $username = ();
> +my $password = ();
> +
> +$url = $opt_server ? $opt_server : "http://bgate.mellanox.com:27017;;
> +$url =~ s/http:\/\///;
> +$username = $opt_username ? $opt_username : "admin";
> +$password = $opt_password ? $opt_password : "";
> +
> +my %conf = ('url' => "$url\/client",
> +'username' => $username,
> +'password' => $password
> +);
> +
> +if ($opt_help)
> +{
> +my $action = '';
> + 
> +$action = 'ping' if ($opt_ping);
> +$action = 'upload' if ($opt_upload);
> +$action = 'query' if ($opt_query);
> +$action = 'view' if ($opt_view);
> +$action = 'admin' if ($opt_admin);
> +
> +help($action);
> +
> +exit;
> +}
> +elsif ($opt_ping)
> +{
> + #ping( \%conf ); 
> + #print $url," url\n";
> + my $conn = MongoDB::Connection->new(host => $url );
> + if($conn != 0)
> + {
> + print"\n\nping: success\n\n";
> + }
> +}
> +elsif ($opt_upload)
> +{
> +if ($#opt_data < 0) 
> +{
> +help('upload');
> +}
> + my @data = split(/,/,join(',',@opt_data)) if (@opt_data);
> + my @raw = split(/,/,join(',',@opt_raw)) if (@opt_raw);
> +
> +# Check if files existed
> + verify_opt_file( @data );
> + verify_opt_file( @raw );
> +
> + $conf{data} = \@data;
> + $conf{raw} = \@raw;  
> +
> + upload( \%conf ); 
> +}
> +elsif ($opt_query)
> +{
> +my $gql = ();
> +if ($opt_gqls) 
> +{
> +$gql = $opt_gqls;
> +}
> +

[MTT devel] Broken MTT trunk

[Jeff Squyres (jsquyres)]

It looks like some of my recent commits have broken the trunk.  I'll be looking 
at this today. 

But it might be safest for those running off the trunk to update to a few days 
ago. 

Sent from my phone. No type good. 

Re: [MTT devel] [MTT svn] svn:mtt-svn r1433

[Jeff Squyres]

> -$test_results->{$this_np} =
> -_run_one_np($install_dir, $run, $mpi_details, $this_np,
> -$force);
> +last
> +   if 
> ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> }
> }
> +
> +last
> +   if ($MTT::Globals::Internals->{is_stopped_on_break_threshold});
> ++$test_count;
> 
> # Write out the "to be saved" test run results
> MTT::Test::SaveRuns($runs_data_dir);
> -
> +
> $MTT::Test::Run::mpi_details = $save_run_mpi_details;
> 
> # Output a progress bar
> @@ -247,6 +259,7 @@
> 
> MTT::Reporter::QueueSubmit();
> }
> +
> }
> 
> sub _run_one_np {
> @@ -290,16 +303,30 @@
> foreach my $e (@$execs) {
> # See if we're supposed to terminate.
> last
> -if (MTT::Util::time_to_terminate());
> +if (MTT::Util::time_to_terminate());
> +
> _run_one_test($install_dir, $run, $mpi_details, $e, $name,
> -  $variant++, $force);
> +$variant++, $force);
> +
> +last
> +if (MTT::Util::check_break_threshold(
> +$test_results_count_global,
> +$break_threshold,
> +$MTT::Globals::Internals->{total_tests_counter})
> +);
> }
> }
> -
> +last
> +if (MTT::Util::check_break_threshold(
> +$test_results_count_global,
> +$break_threshold,
> +$MTT::Globals::Internals->{total_tests_counter})
> +);
> +
> $MTT::Test::Run::test_argv = undef;
> }
> }
> -
> +
> $MTT::Test::Run::test_np = undef;
> }
> 
> @@ -457,6 +484,13 @@
> $test_results_count->{$report->{test_result}}++ 
> if (exists($report->{test_result}));
> 
> +$test_results_count_global->{$report->{test_result}}++
> +if (exists($report->{test_result}));
> +
> +$test_results_count_global->{MTT::Values::TIMED_OUT_OR_FAIL}++
> +if (exists($report->{test_result}) && 
> +(MTT::Values::FAIL == $report->{test_result} || 
> MTT::Values::TIMED_OUT == $report->{test_result}));
> +
> # If there is an after_each step, run it
> $ENV{MTT_TEST_RUN_RESULT_MESSAGE} =
> (MTT::Values::PASS == $report->{test_result} ? "passed" :
> 
> Modified: trunk/lib/MTT/Util.pm
> ==
> --- trunk/lib/MTT/Util.pm (original)
> +++ trunk/lib/MTT/Util.pm 2012-01-25 06:02:47 EST (Wed, 25 Jan 2012)
> @@ -205,6 +205,12 @@
> 
> if (($count->{$result} / $total) > $threshold->{$result}) {
> Verbose("--> Threshold ($per) exceeded for \"$result_label\": 
> $count->{$result} out of $total.\n");
> +$MTT::Globals::Internals->{is_stopped_on_break_threshold} = 
> "true";
> +$MTT::Globals::Internals->{stopped_on_break_threshold_message} = 
> "--> Threshold ($per) exceeded for \"$result_label\": $count->{$result} out 
> of $total.\n";
> +print STDOUT "--> Threshold ($per) exceeded for 
> \"$result_label\": $count->{$result} out of $total.\n";
> +if ($MTT::Globals::Internals->{is_stopped_on_break_threshold}){
> +print STDOUT "0xdeadbeef: it works";
> +}
> return 1;
> }
> }
> 
> Modified: trunk/lib/MTT/Values.pm
> ==
> --- trunk/lib/MTT/Values.pm   (original)
> +++ trunk/lib/MTT/Values.pm   2012-01-25 06:02:47 EST (Wed, 25 Jan 2012)
> @@ -45,6 +45,7 @@
> PASS => 1,
> SKIPPED => 2,
> TIMED_OUT => 3,
> +TIMED_OUT_OR_FAIL =>4,
> };
> 
> # Map to human-readable English labels
> @@ -53,7 +54,7 @@
> $result_messages->{MTT::Values::PASS}  = "pass";
> $result_messages->{MTT::Values::TIMED_OUT} = "timeout";
> $result_messages->{MTT::Values::SKIPPED}   = "skipped";
> -
> +$result_messages->{MTT::Values::TIMED_OUT_OR_FAIL} = "timeout_and_fail";
> # current $ini and $section parameters (we use it in funclets)
> our $evaluate_string_ini;
> our $evaluate_string_section;
> ___
> mtt-svn mailing list
> mtt-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] [MTT svn] svn:mtt-svn r1377

[Jeff Squyres]

Ok, that's what I thought, but your comment doesn't seem to match this:

# mtt_switch(@np@, 9, "return1", 100, return2", "default", 0);

You have an extra "return" in there, and the 5th argument doesn't seem to be 
quoted properly.



On Jan 3, 2011, at 8:21 AM, Mike Dubman wrote:

> Hi,
> it is c-style *switch* replacement, to simplify statements like this:
> 
> transport   =  (('@cycle@', '1'), '@btl_openib@' ,\
>   (('@cycle@', '2'), '@btl_openib@',\
> (('@cycle@', '3'), '@btl_eth10g@',\
>   (('@cycle@', '4'), '@btl_eth10g@',\
> (('@cycle@', '3a'), '@btl_eth10g@ -x 
> custom_noswitchconnect=1',\
>   (('@cycle@', '4a'), '@btl_eth10g@ -x 
> custom_noswitchconnect=1',\
> (('@cycle@', '5'), '@btl_eth10g@',\
>   (('@cycle@', '6'), '@btl_eth10g@',\
> (('@cycle@', '7'), '@btl_eth10g@',\
>   (('@cycle@', '8'), 
> '@btl_eth10g@',\
> (('@cycle@', '11'), 
> '@btl_eth10g@',\
>   (('@cycle@', '12'), 
> '@btl_eth10g@',\
>'@btl_rdmaoe@'\
>  )\
>)\
>  )\
>)\
>  )\
>)\
>  )\
>)\
>  )\
>)\
> )\
>   )
> 
> which can be rewritten as:
> 
> transport = mtt_switch(@cycle@, 1, @btl_openib@, 2, @btl_eth10g@, ...)
> 
> Will update wiki as well.
> 
> 
> Regards
> 
> 
> 
> On Mon, Jan 3, 2011 at 2:54 PM, Jeff Squyres <jsquy...@cisco.com> wrote:
> Mike --
> 
> Can you explain this one a little?  I don't understand the example you gave 
> in the comment.
> 
> Also, are you adding all the new funclets to the wiki documentation?
> 
> 
> 
> On Dec 29, 2010, at 3:52 AM, mi...@osl.iu.edu wrote:
> 
> > Author: miked
> > Date: 2010-12-29 03:52:24 EST (Wed, 29 Dec 2010)
> > New Revision: 1377
> > URL: https://svn.open-mpi.org/trac/mtt/changeset/1377
> >
> > Log:
> > added mtt_switch to simplify if-then-else cases in ini files
> >
> > Text files modified:
> >   trunk/lib/MTT/Values/Functions.pm |21 +
> >   1 files changed, 21 insertions(+), 0 deletions(-)
> >
> > Modified: trunk/lib/MTT/Values/Functions.pm
> > ==
> > --- trunk/lib/MTT/Values/Functions.pm (original)
> > +++ trunk/lib/MTT/Values/Functions.pm 2010-12-29 03:52:24 EST (Wed, 29 Dec 
> > 2010)
> > @@ -3163,4 +3163,25 @@
> >   return $x->{result_stdout};
> > }
> >
> > +#
> > +# Poor man switch statement
> > +# Example: mtt_switch(@np@, 9, "return1", 100, return2", "default", 0);
> > +#
> > +
> > +sub mtt_switch
> > +{
> > +my ($var, %cases) = @_;
> > +
> > +    if ($cases{$var}) {
> > +return $cases{$var};
> > +}
> > +
> > +if ($cases{'default'}) {
> > +    return $cases{'default'};
> > +}
> > +
> > +Debug("ERROR: Not found case for $var\n");
> > +}
> > +
> > +
> > 1;
> > ___
> > mtt-svn mailing list
> > mtt-...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
> 
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] Best way to run ftb_database_server and ftb_agent

[Jeff Squyres]

On Nov 9, 2010, at 1:00 AM, DongInn Kim wrote:

> No, I did not know that it should be added in the MPI Get phase.

You have to call all the phases, even if they don't "do" anything.  That's why 
we have no-op / alreadyinstalled versions of plugins, for example.  Each phase 
sets up data structures that are used by subsequent phases.

> OK, I tried to add it the MPI Get phase and mpi_details are recognized but I 
> could not have "Test Run" phase run the scripts in before_any_exec and 
> after_all_exec.

What exactly do you have in your ini file again for these fields?

I have this in an old ini file -- it *used* to work (when launching MPICH2 and 
Intel MPI jobs through MTT):

before_any_exec = < $file
# Add localhost if it's not in there (e.g., srun -A)
local=`grep $h $file`
touch /tmp/mtt-mpiexec-options.$SLURM_JOBID
if test "$local" = ""; then
   echo $h >> $file
   echo -nolocal > /tmp/mpiexec-options.$SLURM_JOBID
fi
num=`wc -l $file | awk '{ print $1 }'`
mpdboot -n $num -r ssh --verbose --file=$file
mpdtrace
rm -f $file
EOF

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] Best way to run ftb_database_server and ftb_agent

[Jeff Squyres]

Are you executing an MPI Get section?

I believe that the mpi_details section is filled in via the MPI Get phase and 
then propagated down through all the other phases (i.e., each of the other 
phases looks a the way back to their corresponding mpi get phase to find 
the mpi_details value).


On Nov 8, 2010, at 5:16 PM, DongInn Kim wrote:

> Jeff, thank you.
> 
> BTW, I have looked at the ompi-core-perf-testing.ini file which seems to have 
> used the mpi detail sections and I tried to use it in our ftb.ini file but I 
> still get the same warning message.
> 
> *** Test Run phase starting
>>> Test Run [ftb]
>>> Running with [ftb-nightly-trunk] / [0.6.2] / [platform]
> *** WARNING: Unable to find MPI details section for [MPI Install: platform;
>skipping
> *** Run test phase complete
>>> Phase: Test Run
>   Started:   Mon Nov  8 17:10:30 2010
>   Stopped:   Mon Nov  8 17:10:31 2010
>   Elapsed:   00:00:01 0.02u 0.06s
>   Total elapsed: 00:00:01 0.02u 0.06s
>>> Phase: Trim
>   Started:   Mon Nov  8 17:10:31 2010
>   Stopped:   Mon Nov  8 17:10:31 2010
>   Elapsed:   00:00:00 0.00u 0.00s
>   Total elapsed: 00:00:01 0.02u 0.06s
> *** Reporter finalizing
> *** Reporter finalized
> 
> 
> Here is the entry in the new ftb.ini file.
> #--   
> 
> 
> [MPI Details: platform] 
> 
> # Need a before_any_exec step to test all the FTB example tests   
> 
> before_any_exec = < install_dir=_prefix_pretty() 
> 
> ftb_server_daemon="$install_dir/sbin/ftb_database_server" 
> 
> ftb_agent_daemon="$install_dir/sbin/ftb_agent"
> 
> $ftb_server_daemon &
> $ftb_agent_daemon 
> EOF
> 
> after_all_exec = < 
> ftb_db_pid=`pgrep ftb_database_server`
> kill $ftb_db_pid
> ftb_agent_pid=`pgrep ftb_agent`   
> 
> kill $ftb_agent_pid
> EOT
> 
> #------
> 
> I have tried to replace "platform" with "FTB" in "[MPI Details: platform]" 
> but it still did not work.
> 
> Any helps on this?
> 
> Regards,
> 
> 
> On 11/8/10 3:42 PM, Jeff Squyres wrote:
>> Sorry for jumping in late -- been swamped recently...
>> 
>> In the MPI details section, there's 4 fields that should let you do what you 
>> want.
>> 
>> before_any_exec -- run once before all the tests in a given Test Run
>> before_each_exec -- run once before every single exec (including all 
>> variants)
>> after_each_exec -- run after after every single exec (include all variants)
>> after_all_exec -- run after all tests in a given Test Run section have 
>> completed
>> 
>> So you can use the before_any_exec / after_all_exec to launch the daemons 
>> once at the beginning and then take them down, or you can use 
>> before_each_exec / after_each_exec to launch the daemons before each test 
>> and then take them down at the end of that test.
>> 
>> I'm assuming that the *each* variants will cause your tests to run much 
>> slower.
>> 
>> I see that we don't have an MPI Details section on the wiki describing these 
>> parameters.  Sorry!  :-(
>> 
>> 
> 
> 
> -- 
> - DongInn
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] Best way to run ftb_database_server and ftb_agent

[Jeff Squyres]

:31 AM, DongInn Kim wrote:
>> Josh, thanks but actually I don't know how to set the base directory of the 
>> test suite in the Test Get/Build phase. Is it setup in the ini file or 
>> somewhere? and how?
>> 
>> Regards,
>> 
>> On 11/5/10 11:32 AM, Joshua Hursey wrote:
>>> 
>>> On Nov 5, 2010, at 10:11 AM, DongInn Kim wrote:
>>> 
>>>> Josh, I have another question.
>>>> How can mtt find the script to run?
>>>>>> exec = ./run-cr-correctness.pl -test ...
>>>> 
>>>> I can write a similar script like run-correctness.pl but if I put my 
>>>> script(e.g., run-ftb-tests.pl) to ftb-tests/iu/ftt/ftb/, how can I make 
>>>> mtt recognize this path and file?
>>> 
>>> The file is in the ompi-tests repo, under:
>>>   ompi-tests/iu/ft/correctness/run-correctness.pl
>>> 
>>> Remember that the Test Run phase will 'cd' into the base directory of the 
>>> test suite that you specify in the Test Get/Build phase. So you can 
>>> reference a relative path with respect to the test suite. So just put the 
>>> script in with the test suite, and the Test Get phase will grab it for you.
>>> 
>>> -- Josh
>>> 
>>>> 
> 
> -- 
> - DongInn
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] questions about MTT database from HDF

[Jeff Squyres]

Yep; I mentioned your GDS-backend work to the HDF folks.  But your email is 
much more detailed than what I mentioned -- thanks!


On Nov 7, 2010, at 1:02 AM, Mike Dubman wrote:

> 
> Hi,
> Also, there is an MTT option to select Google Datastore as a storage backend 
> for mtt results.
> 
> 
> Pro:
>  - your data is stored in the Google`s cloud
>  - You can access your data from scripts
>  - You can create a custom UI for you data visualization
>  - You can use Google`s default datastore querying tools 
>  - seamless integration with mtt
>  - No need in DBA services 
>  - There are some simple report scripts to query data and generate Excel files
>  - You can define custom dynamic DB fields and associate it with your data
>  - You can define security policy/permissions for your data
> 
> Cons:
>  - No UI (mtt default UI works with sql backend only)
> 
> regards
> Mike
> 
> On Thu, Nov 4, 2010 at 11:08 PM, Quincey Koziol <koz...@hdfgroup.org> wrote:
> Hi Josh!
> 
> On Nov 4, 2010, at 8:30 AM, Joshua Hursey wrote:
> 
> >
> > On Nov 3, 2010, at 9:10 PM, Jeff Squyres wrote:
> >
> >> Ethan / Josh --
> >>
> >> The HDF guys are interested in potentially using MTT.
> >
> > I just forwarded a message to the mtt-devel list about some work at IU to 
> > use MTT to test the CIFTS FTB project. So maybe development between these 
> > two efforts can be mutually beneficial.
> >
> >> They have some questions about the database.  Can you guys take a whack at 
> >> answering them?  (be sure to keep the CC, as Elena/Quincey aren't on the 
> >> list)
> >>
> >>
> >> On Nov 3, 2010, at 1:29 PM, Quincey Koziol wrote:
> >>
> >>> Lots of interest here about MTT, thanks again for taking time to demo 
> >>> it and talk to us!
> >>
> >> Glad to help.
> >>
> >>> One lasting concern was the slowness of the report queries - what's 
> >>> the controlling parameter there?  Is it the number of tests, the size of 
> >>> the output, the number of configurations of each test, etc?
> >>
> >> All of the above.  On a good night, Cisco dumps in 250k test runs to the 
> >> database.  That's just a boatload of data.  End result: the database is 
> >> *HUGE*.  Running queries just takes time.
> >>
> >> If the database wasn't so huge, the queries wouldn't take nearly as long.  
> >> The size of the database is basically how much data you put into it -- so 
> >> it's really a function of everything you mentioned.  I.e., increasing any 
> >> one of those items increases the size of the database.  Our database is 
> >> *huge* -- the DB guys tell me that it's lots and lots of little data (with 
> >> blobs of stdout/stderr here an there) that make it "huge", in SQL terms.
> >>
> >> Josh did some great work a few summers back that basically "fixed" the 
> >> speed of the queries to a set speed by effectively dividing up all the 
> >> data into month-long chunks in the database.  The back-end of the web 
> >> reporter only queries the relevant month chunks in the database (I think 
> >> this is a postgres-specific SQL feature).
> >>
> >> Additionally, we have the DB server on a fairly underpowered machine that 
> >> is shared with a whole pile of other server duties (www.open-mpi.org, 
> >> mailman, ...etc.).  This also contributes to the slowness.
> >
> > Yeah this pretty much sums it up. The current Open MPI MTT database is 141 
> > GB, and contains data as far back as Nov. 2006. The MTT Reporter takes some 
> > of this time just to convert the raw database output into pretty HTML (it 
> > is currently written in PHP). At the bottom of the MTT Reporter you will 
> > see some stats on where the Reporter took most of its time.
> >
> > How long the Reporter took total to return the result is:
> >  Total script execution time: 24 second(s)
> > How long just the database query took is reported as:
> >  Total SQL execution time: 19 second(s)
> >
> > We also generate an overall contribution graph which is also linked at the 
> > bottom to give you a feeling of the amount of data coming in every 
> > day/week/month.
> >
> > Jeff mentioned the partition tables work that I did a couple summers ago. 
> > The partition tables help quite a lot by partitioning the data into week 
> > long chunks so shorter date ranges will be faster than longer date ranges 
> > since they pull a smaller table with respect to all of the data 

[MTT devel] questions about MTT database from HDF

[Jeff Squyres]

Ethan / Josh --

The HDF guys are interested in potentially using MTT.  They have some questions 
about the database.  Can you guys take a whack at answering them?  (be sure to 
keep the CC, as Elena/Quincey aren't on the list)


On Nov 3, 2010, at 1:29 PM, Quincey Koziol wrote:

>   Lots of interest here about MTT, thanks again for taking time to demo 
> it and talk to us!

Glad to help.

>   One lasting concern was the slowness of the report queries - what's the 
> controlling parameter there?  Is it the number of tests, the size of the 
> output, the number of configurations of each test, etc?  

All of the above.  On a good night, Cisco dumps in 250k test runs to the 
database.  That's just a boatload of data.  End result: the database is *HUGE*. 
 Running queries just takes time.

If the database wasn't so huge, the queries wouldn't take nearly as long.  The 
size of the database is basically how much data you put into it -- so it's 
really a function of everything you mentioned.  I.e., increasing any one of 
those items increases the size of the database.  Our database is *huge* -- the 
DB guys tell me that it's lots and lots of little data (with blobs of 
stdout/stderr here an there) that make it "huge", in SQL terms. 

Josh did some great work a few summers back that basically "fixed" the speed of 
the queries to a set speed by effectively dividing up all the data into 
month-long chunks in the database.  The back-end of the web reporter only 
queries the relevant month chunks in the database (I think this is a 
postgres-specific SQL feature).

Additionally, we have the DB server on a fairly underpowered machine that is 
shared with a whole pile of other server duties (www.open-mpi.org, mailman, 
...etc.).  This also contributes to the slowness.

> For example, each HDF5 build includes on the order of 100 test executables, 
> and we run 50 or so configurations each night.  How would that compare with 
> the OpenMPI test results database?

Good question.  I'm CC'ing the mtt-devel list to see if Josh or Ethan could 
comment on this more intelligently than me -- they did almost all of the 
database work, not me.

I'm *guessing* that it won't come anywhere close to the size of the Open MPI 
database (we haven't trimmed the data in the OMPI database since we started 
gathering data in the database several years ago).

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTToGDS

[Jeff Squyres (jsquyres)]

Ok. I'll try to look at this as well - but no promises about when I can do 
so... 

-jms 
Sent from my PDA. No type good.



From: Igor Ivanov <igor.iva...@itseez.com> 
To: Igor Ivanov <igor.iva...@itseez.com> 
Cc: b...@argus-cv.com <b...@argus-cv.com>; Igor Ivanov 
<igor.iva...@argus-cv.com>; Development list for the MPI Testing Tool 
<mtt-de...@open-mpi.org>; mtt-devel-boun...@open-mpi.org 
<mtt-devel-boun...@open-mpi.org>; Mike Dubman <mi...@voltaire.com>; Jeff 
Squyres (jsquyres) 
Sent: Thu Mar 04 01:31:16 2010
Subject: Re: [MTT devel] MTToGDS 




Igor Ivanov wrote: 

I considered possibility to use cookie when issue was found out. But 
looking google documentation I could not understand if it solved this issue. So 
it require additional investigation that I do not have now. I will look in this 
way closer but current quick solutions were suggested in mail.
Now we have information about issue and know ways to workaround them.
    
Igor

Jeff Squyres wrote: 

Yoinks.

Alternatively, doesn't a Google login return a cookie of some 
flavor that is valid for a long period of time (somewhere between 1 day and 2 
weeks)?  Can't we keep/cache that cookie down in the MTT client and use it for 
subsequent data submissions until the cookie expires and we have to login again?


On Feb 27, 2010, at 8:30 AM, Igor Ivanov wrote:

  

Description:
Issue arises during submitting data frequently. We can 
get failure during data submitting with authentication error.

Reason:
Google allows a failure response on The ClientLogin 
authorization process with a CAPTCHA challenge means that Google has decided, 
for whatever reason, that additional security measures should be taken. This 
response is accompanied by a CAPTCHA image URL and a token representing the 
specific CAPTCHA challenge.
I do not see way to organize customer input in this 
case.

Detail information can be found at:

http://code.google.com/intl/en-EN/apis/accounts/docs/AuthForInstalledApps.html

Possible solutions:
1. catch error condition on server side and return 
status 503: 'Service Unavailable';
In this case client can organize processing of this 
failure (it is possible that sleeping for some time could help)
2. catch error condition on server side and accept 
authentication by correct username only w/o real verification;
3. rollback to previous scheme;


Igor

Igor Ivanov wrote:


Hi Jeff,

I am sending patch that enable google account 
approach to submit data to MTT GDS.
It also has the fix to a bug in displaying 
table as respond to bquery.pl --view (It has not been committed yet to MTT 
trunk).

As for question relating "how does one develop 
..." that source information can be found at following location as: 
http://svn.open-mpi.org/svn/mtt/trunk/docs/gds/VBench_GDS_Setup.doc.
In case you make a resolve to accept patch I am 
sending next steps should be done:

1. update application on server side using 
instruction in VBench_GDS_Setup.doc (topic 4 "Installation")
example: appcfg.py update /
2. change version on 
https://appengine.google.com/deployment?_id=open-mpi-mtt_id=1.337140739868725607
 from 1 to 2 (make default)
note: after this operation all users that 
attempt to submit data using previous scheme of authentication will get failure 
respond.
3. go to open-mpi-mtt and add new users with 
google account


Regards,
Igor
        
Jeff Squyres wrote:
  

Great -- many thanks!

  

Re: [MTT devel] MTToGDS

[Jeff Squyres (jsquyres)]

Fair enough (the appspot is quit limited to admin). 

But the next time we edit it, if we could add some kind of printf about "you 
are signed in with a google account that does not have access to this portion 
of the app spot; please re-login as an authorized user" or something simple 
like that, that would be great. 

That's all I was asking about - not developing more capabilities of the admin 
side of the appspot. 

-jms 
Sent from my PDA. No type good.



From: Igor Ivanov <igor.iva...@itseez.com> 
To: Igor Ivanov <igor.iva...@itseez.com> 
Cc: b...@argus-cv.com <b...@argus-cv.com>; Igor Ivanov 
<igor.iva...@argus-cv.com>; Development list for the MPI Testing Tool 
<mtt-de...@open-mpi.org>; yift...@voltaire.com <yift...@voltaire.com>; Mike 
Dubman <mi...@voltaire.com>; Jeff Squyres (jsquyres) 
Sent: Thu Mar 04 01:30:56 2010
Subject: Re: [MTT devel] MTToGDS 




Igor Ivanov wrote: 

Hi Jeff,

Look  at my comments below.

Note: be aware that my mail has been changed to itseez.com domain.

Igor

Jeff Squyres wrote: 

On Feb 16, 2010, at 10:19 AM, Igor Ivanov wrote:

  

I am sending patch that enable google account approach 
to submit data to MTT GDS.
It also has the fix to a bug in displaying table as 
respond to bquery.pl --view (It has not been committed yet to MTT trunk).



Thanks guys!  And sorry for my long lack of response - I was 
working in a window of availability for MTT stuff before, and then that window 
closed and I got sucked into other high priority things.  I have another small 
window of availability for MTT now...

It looks like Mike D. committed this stuff to SVN already, 
right?  If so, great!

  

As for question relating "how does one develop ..." 
that source information can be found at following location as: 
http://svn.open-mpi.org/svn/mtt/trunk/docs/gds/VBench_GDS_Setup.doc.



Ok, I'll dig into that.

One thing that is quite confusing is the sign in to 
http://open-mpi-mtt.appspot.com/.  When you go there, you get a "Sign in or 
register" link.  You click that and get to a Google Accounts login.  I used 
openmpi.ci...@gmail.com and its associated password, but then I'm bounced back 
to the "Sign in or register" link.

Only when I login as open...@gmail.com do I actually get beyond 
the "Sign in or register" link.

Does this mean that openmpi.ci...@gmail.com does not have login 
privlidges on the open-mpi-mtt appspot?  If so, can we add a better error 
message than this?  It's very confusing -- because you *are* apparently signing 
in to a google account properly, but then you just get the "Sign in or 
register" link again.
  

[ii] As I mentioned in previous mails current form of web-site suits 
administrating purpose mostly (to fit full multiuser access it should be 
provided additional
operations). So I knowingly limited admission for administrator only as 
"openmpi" to avoid additional questions in multiuser usage. 
  

  

In case you make a resolve to accept patch I am sending 
next steps should be done:

1. update application on server side using instruction 
in VBench_GDS_Setup.doc (topic 4 "Installation")
example: appcfg.py update /
2. change version on 
https://appengine.google.com/deployment?_id=open-mpi-mtt_id=1.337140739868725607
 from 1 to 2 (make default)
note: after this operation all users that attempt to 
submit data using previous scheme of authentication will get failure respond.
3. go to open-mpi-mtt and add new users with google 
account



It looks like this was all done already -- probably because I 
took so long to reply.

Thanks!

  



__ Information from ESET NOD32 Antivirus, version of virus 
signature database 4913 (20100303) __

The message was checked by ESET NOD32 Antivirus.

http://www.esetnod32.ru


__ Information from ESET NOD32 Antivirus, version of virus 
signature database 4913 (20100303) __

The message was checked by ESET NOD32 Antivirus.

http://www.eset

Re: [MTT devel] MTToGDS

[Jeff Squyres]

Yoinks.

Alternatively, doesn't a Google login return a cookie of some flavor that is 
valid for a long period of time (somewhere between 1 day and 2 weeks)?  Can't 
we keep/cache that cookie down in the MTT client and use it for subsequent data 
submissions until the cookie expires and we have to login again?


On Feb 27, 2010, at 8:30 AM, Igor Ivanov wrote:

> Description:
> Issue arises during submitting data frequently. We can get failure during 
> data submitting with authentication error.
> 
> Reason:
> Google allows a failure response on The ClientLogin authorization process 
> with a CAPTCHA challenge means that Google has decided, for whatever reason, 
> that additional security measures should be taken. This response is 
> accompanied by a CAPTCHA image URL and a token representing the specific 
> CAPTCHA challenge.
> I do not see way to organize customer input in this case.
> 
> Detail information can be found at:
> http://code.google.com/intl/en-EN/apis/accounts/docs/AuthForInstalledApps.html
> 
> Possible solutions:
> 1. catch error condition on server side and return status 503: 'Service 
> Unavailable';
> In this case client can organize processing of this failure (it is possible 
> that sleeping for some time could help)
> 2. catch error condition on server side and accept authentication by correct 
> username only w/o real verification;
> 3. rollback to previous scheme;
> 
> 
> Igor
> 
> Igor Ivanov wrote:
>> Hi Jeff,
>> 
>> I am sending patch that enable google account approach to submit data to MTT 
>> GDS.
>> It also has the fix to a bug in displaying table as respond to bquery.pl 
>> --view (It has not been committed yet to MTT trunk).
>> 
>> As for question relating "how does one develop ..." that source information 
>> can be found at following location as: 
>> http://svn.open-mpi.org/svn/mtt/trunk/docs/gds/VBench_GDS_Setup.doc.
>> In case you make a resolve to accept patch I am sending next steps should be 
>> done:
>> 
>> 1. update application on server side using instruction in 
>> VBench_GDS_Setup.doc (topic 4 "Installation")
>> example: appcfg.py update /
>> 2. change version on 
>> https://appengine.google.com/deployment?_id=open-mpi-mtt_id=1.337140739868725607
>>  from 1 to 2 (make default)
>> note: after this operation all users that attempt to submit data using 
>> previous scheme of authentication will get failure respond.
>> 3. go to open-mpi-mtt and add new users with google account
>> 
>> 
>> Regards,
>> Igor
>> 
>> Jeff Squyres wrote:
>>> Great -- many thanks!
>>> 
>>> On Feb 12, 2010, at 12:32 PM, Igor Ivanov wrote:
>>> 
>>>   
>>> 
>>>> Hi Jeff,
>>>> 
>>>> I have done changes related google account support but not tested them 
>>>> well.
>>>> I will try to send them on Monday.
>>>> 
>>>> Regards,
>>>> Igor
>>>> 
>>>> Jeff Squyres wrote:
>>>> 
>>>> 
>>>>> On Feb 10, 2010, at 9:09 AM, Igor Ivanov wrote:
>>>>> 
>>>>>   
>>>>> 
>>>>>   
>>>>> 
>>>>>>> I took a swipe at doing this (totally not tested; how does one 
>>>>>>> develop/test this stuff?).  I know just a tiny bit of python, but the 
>>>>>>> code was fairly readable.  Please see the attached patch -- is it 
>>>>>>> anywhere close to correct?
>>>>>>> 
>>>>>>>   
>>>>>>> 
>>>>>>>   
>>>>>>> 
>>>>>> [II] It seems close but you forget about bquery.pl that allows to add a 
>>>>>> new user and related handler (processes bquery.pl --admin) on 
>>>>>> gds/main.py at least.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> Oh, yikes -- good catch.  I'll look into that...
>>>>> 
>>>>> How does one develop / test / debug / deploy changes to this stuff?
>>>>> 
>>>>>   
>>>>> 
>>>>>   
>>>>> 
>>>> __ Information from ESET NOD32 Antivirus, version of virus 
>>>> signature database 4861 (20100212) __
>>>> 
>>>> The message was checked by ESET NOD32 Antivirus.
>>>> 
>>>> 
>>>> http://www.esetnod32.ru
>>>> 
>>&g

Re: [MTT devel] MTT GDS -- one more...

[Jeff Squyres]

On Feb 21, 2010, at 5:52 AM, Mike Dubman wrote:

> > > 2. In reading through the Google Appengine docs, the GDS stuff looks like
> > we mainly can access the data through GQL.  I don't see any mention of doing
> > map/reduce kinds of computations (Ethan and I were talking on the phone
> > today about MTT Appengine possibilities).  I'm new to all this stuff, so
> > it's quite possible that a) I missed it, or b) I just don't understand what
> > I'm seeing/reading yet.  Or does GQL do map/reduce on the back end to do its
> > magic?  Is GQL the main/only way we have to access GDS?
> >
> > As far as I and Igor know there are no way of doing Map/Reduce with GDS. And
> > GQL (or filters which is practically synonym) is the main and only way to
> > access GDS data.
> 
> btw, afaik - the gds is using mapreduce explicitly on the backend - one needs 
> do nothing in order to ENABLE it: just use their API and the framework will 
> do the rest.

Hm; ok.  So they only give us limited access to map reduce -- via GQL?  (i.e., 
we can only do what GQL allows us to do)

For example, I'd love to be able to specify my own reductions in order to do 
some real data mining, parsing through the data, etc.  Specifically, I'd like 
to be able to exploit the whole "bring the computation to the data" philosophy 
of map reduce, because MTT data can be *huge*.  But I don't see how to do this 
with GQL...?

Am I missing something?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] More GDS questions

[Jeff Squyres]

On Feb 12, 2010, at 10:02 AM, Igor Ivanov wrote:

> You touched a sore point. App engine forums are in filled  the questions as 
> yours. 
> I can not know clear answer now.

Ok, bummer.  :-)

>> 2. I'm still looking into the perl syntax error that caused my Big Submit to 
>> GDS to fail.  But looking at the Google logs, it looks like at least *some* 
>> of my test run results made it up to GDS.  There was a BIG spike in CPU 
>> usage (3.2 hours of CPU time!) when it submitted -- see the attached CPU 
>> usage graph from the apps dashboard.
>> 
>> Does anyone know why it takes so much CPU just to submit data to GDS?  3.2 
>> CPU hours is a LOT!
>> 
>> It makes me a bit concerned that only part of a single Cisco MTT run submit 
>> checked through almost half of our daily CPU quota (6.5 CPU hours/day).  Is 
>> there any way to reduce the amount of CPU necessary just to submit data?

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] MTT GDS -- one more...

[Jeff Squyres]

Heh... even more questions...

(BTW, Ethan and I have asked s many questions that if it helps, I can setup 
a webex and we can all discuss this in person rather than via 1,000,000 
annoying emails from us.  :-)  Webex can call you; no one will need to pay for 
an international call)

1. It looks like the main benefits of using the Google App Engine -- 
specifically for MTT -- is that we can use the GDS and/or we can host an 
application on their web servers.  Is that correct?

2. In reading through the Google Appengine docs, the GDS stuff looks like we 
mainly can access the data through GQL.  I don't see any mention of doing 
map/reduce kinds of computations (Ethan and I were talking on the phone today 
about MTT Appengine possibilities).  I'm new to all this stuff, so it's quite 
possible that a) I missed it, or b) I just don't understand what I'm 
seeing/reading yet.  Or does GQL do map/reduce on the back end to do its magic? 
 Is GQL the main/only way we have to access GDS?

3. Is there a reason that MTTGDS.pm doesn't use the python API to directly talk 
to GDS?  I.e., what is the rationale for using a web app on appengine?  Is the 
web app doing stuff that we can't do at the client?  Ditto for bquery.pl and 
breport.pl.  (these questions are partially fueled by my curiosity and concern 
about why we're using so much CPU at Google)

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] More GDS questions

[Jeff Squyres]

Igor et al. -- 

1. I'm not sure you saw Ethan's and my posts from the past day or so about GDS 
on the mtt-devel list; it just occurred to me that I don't know if you're 
members of the list or not.  We've posted a few questions and comments that you 
may not have received if you're not on the list:

http://www.open-mpi.org/community/lists/mtt-devel/2010/02/index.php

2. I'm still looking into the perl syntax error that caused my Big Submit to 
GDS to fail.  But looking at the Google logs, it looks like at least *some* of 
my test run results made it up to GDS.  There was a BIG spike in CPU usage (3.2 
hours of CPU time!) when it submitted -- see the attached CPU usage graph from 
the apps dashboard.

Does anyone know why it takes so much CPU just to submit data to GDS?  3.2 CPU 
hours is a LOT!

It makes me a bit concerned that only part of a single Cisco MTT run submit 
checked through almost half of our daily CPU quota (6.5 CPU hours/day).  Is 
there any way to reduce the amount of CPU necessary just to submit data?

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] GDS errors

[Jeff Squyres]

Looking in the appspot dashboard, I see a bunch of errors when Cisco tried to 
submit test run data.  There's a few random errors, but a bunch that look like 
what I pasted below.  How do I diagnose this further?  Clearly, some field is 
too long -- how do I find out which one?

-
• 128.107.241.170 - - [11/Feb/2010:00:51:21 -0800] "POST /client 
HTTP/1.1" 500 1972 - "MPI Test MTTGDS Reporter,gzip(gfe)" 
"open-mpi-mtt.appspot.com"
• E02-11 12:51AM 21.241
Property data_message_size is 667 bytes long; it must be 500 or less. Consider 
Text instead, which can store strings of any length.
Traceback (most recent call last):
  File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", 
line 509, in __call__
handler.post(*groups)
  File "/base/data/home/apps/open-mpi-mtt/1.337140739868725607/main.py", line 
961, in post
status = self._submit();
  File "/base/data/home/apps/open-mpi-mtt/1.337140739868725607/main.py", line 
485, in _submit
test_run_phase.put()
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 
801, in put
self._populate_internal_entity()
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 
779, in _populate_internal_entity
self._entity = self._populate_entity(_entity_class=_entity_class)
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 
839, in _populate_entity
self._to_entity(entity)
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 
1465, in _to_entity
entity[key] = value
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 
492, in __setitem__
datastore_types.ValidateProperty(name, value)
  File "/base/python_lib/versions/1/google/appengine/api/datastore_types.py", 
line 1290, in ValidateProperty
prop_validator(name, v)
  File "/base/python_lib/versions/1/google/appengine/api/datastore_types.py", 
line 1181, in ValidatePropertyString
ValidateStringLength(name, value, max_len=_MAX_STRING_LENGTH)
  File "/base/python_lib/versions/1/google/appengine/api/datastore_types.py", 
line 1171, in ValidateStringLength
(name, len(value), max_len))
BadValueError: Property data_message_size is 667 bytes long; it must be 500 or 
less. Consider Text instead, which can store strings of any length.
-

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] 500 Internal Server Error from open-mpi-mtt.appspot.com

[Jeff Squyres]

After looking through the logs, Ethan and I *think* that this was just a query 
that was too large (i.e., it used too much CPU, and therefore it was killed).

Can someone with a little more knowledge than us have a look at the logs and 
let us know if we're right?


On Feb 11, 2010, at 2:05 PM, Ethan Mallove wrote:

> Hi,
> 
> I'm getting a 500 Internal Server Error using bquery.pl.  I can --ping
> successfully:
> 
>   $ client/bquery.pl --ping --server=http://open-mpi-mtt.appspot.com/ 
> --password=x --username=sun
>   Ping is successful.
> 
> But an actual query gets an error:
> 
>   $ client/bquery.pl --server=http://open-mpi-mtt.appspot.com/ 
> --password=x --username=sun --query --gqls="select * from TestRunPhase 
> where status=1" --dir="bquery-test"
>   Error at http://open-mpi-mtt.appspot.com//client
>   500 Internal Server Error
> 
> -Ethan
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
> 


-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

[MTT devel] MTTGDS issues

[Jeff Squyres]

1. Can you guys describe what MTTGDS expects from the performance analyzer 
modules?

I ran a bunch of netpipe results and MTTGDS performance analyzer failed to run 
-- did you guys change the specifications for the performance analyzer modules?

*** WARNING: Could not run module
MTT::Test::Analyze::Performance::NetPipe:PreReport: Undefined
subroutine ::Test::Analyze::Performance::NetPipe::PreReport called
at (eval 335838) line 1.

2. I ran 24+ hours of MTT tests and the MTTGDS reporter failed to submit the 
results.  :-(

*** ERROR: Module aborted: MTT::Reporter::MTTGDS:Finalize: Nested
quantifiers in regex; marked by <-- HERE in m/\s[\S/\\]*mpi2c++ <--
HERE _test.*/ at /home/jsquyres/svn/mtt/lib/MTT/Reporter/MTTGDS.pm line
498.

Some of my INI section names have special characters in them (e.g., "mpi2c++"); 
it looks like this might be what tripped up some regexp.  I'll have a look at 
this one now...

Is there a way to re-submit my data to GDS?

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTToGDS

[Jeff Squyres]

On Feb 10, 2010, at 4:18 AM, Igor Ivanov wrote:

> I believe that it is just a warning and you can use mtt w/o analyzer that 
> allow get additional info from output.

True, it's just a warning.  But it's (very) big and ugly.  :-)  Makes it quite 
difficult to read --verbose output and see if anything actually went wrong.

Is this patch correct?  It checks to see if there is no analyze_module, and if 
so, just returns the form.


Index: lib/MTT/Reporter/MTTGDS.pm
===
--- lib/MTT/Reporter/MTTGDS.pm  (revision 1346)
+++ lib/MTT/Reporter/MTTGDS.pm  (working copy)
@@ -576,6 +576,11 @@

 my $ini= $MTT::Globals::Internals->{ini};
 my $module = $ini->val( "Test run: " . $section, "analyze_module" );
+
+# If there's no analyze module, then just return
+return $form
+if (!$module);
+
 $module = "MTT::Test::Analyze::Performance::$module";
 my $method = "PreReport";
     my @args   = ( $phase, $section, $report );


-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTToGDS

[Jeff Squyres]

On Feb 10, 2010, at 4:12 AM, Igor Ivanov wrote:

>> Is it hard to redirect the appspot lookup to use google account names + 
>> passwords?
>> 
> [II] I believe that it is possible task. It could be done in two ways:
> set google account e-mail in mttdatabase_username key of ini-file
> 1) provide for filling User.username with google account e-mail and change 
> code of User.check_password in file  gds/auth/models.py to with google 
> account verification code
> code example (I have not checked one):

I took a swipe at doing this (totally not tested; how does one develop/test 
this stuff?).  I know just a tiny bit of python, but the code was fairly 
readable.  Please see the attached patch -- is it anywhere close to correct?

User.get_full_name() would still need to be re-done.  How does one fetch Google 
account info like that?

> Keep in mind performance difference between google account verification code 
> and local verification!

Yep -- am not worried about that.  MTT data submits don't have to be super 
speedy.  If a local verification takes (say) .01 second, and a google account 
verification takes 1 second (or even a few seconds), I don't think it'll matter.

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/


use-google-account.patch
Description: Binary data

Re: [MTT devel] MTToGDS

[Jeff Squyres]

On Feb 10, 2010, at 1:45 AM, Mike Dubman wrote:

> The GDS files use libyaml interfaces (there is no explictic 'use Syck'
> in the code). I think it is implicit dependancy of one of the perl
> modules or when you do "yum install libyaml perl-Yaml --> it brings syck
> as well)

It seems to be coming from bquery.pl (I didn't try breport.pl yet) -- if I 
don't have YAML::Syck installed, perl complains that it can't find it in @INC.  
Odd.

> > I'm testing bquery.pl -- unfortunately, I'm behind some
> > proxies in the Cisco lab environment.  I'll see if I can add
> > proxy support...
> 
> It works for us behind the proxy with HTTP_PROXY shell vars.

I committed some changes to bquery.pl last night for this.  The issue is that 
$ENV{http_proxy} is unfortunately overloaded by multiple different apps and 
environments -- some require the value to be of the form:

proxy_host_name[:port]

Others (like LWP) require it to be of the form

scheme://proxy_host_name[:port]

In my environment, I use the former (with no scheme).  So I added some code in 
bquery.pl to check and see if there is no scheme at the beginning of 
$ENV{http_proxy}.  If there isn't, add the same scheme that is at the prefix of 
the URL that we're submitting to.

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] MTToGDS

[Jeff Squyres]

6. Could you send detail info about the issue (ini-file, mtt.log with verbose 
> info and command line), we will look on that.

Let me reproduce and simplify; I was using a fairly complex ini file...  

Thanks!

-- 
Jeff Squyres
jsquy...@cisco.com

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [MTT devel] mtt not working on sles 11up2 perl 5.10.0

[Jeff Squyres]

Hmm.  This is trunk, I assume?  If so, Values.pm:107 is:


my $ret = _eval_func($func_name, $func_args);

# If we got a string back, append the remaining and loop
# around looking for more 
--> if (ref($ret) eq "") {
-

So it's the return of evaluating the function.

Your output says ">>> Using group_reports" , but I don't see a "group_reports" 
in mini.ini...?

I think what I would do here is try to figure out what function it was trying 
to invoke -- perhaps something returned undef instead of a value...?  If so, 
perhaps we should protect that if statement if a check to see if $ret is 
defined...?


On Jan 27, 2010, at 4:36 AM, Mike Dubman wrote:

> 
> 
> Hello guys,
> 
> 
> mtt fails on sles11up2 with perl version 5.10.0 but works on other distros as 
> a charm.
> 
> The same minimalistic ini file which works on other distro`s fails on sles 
> with error:
> 
> >> Test Run [osu]
> >> Running with [open mpi] / [1.3.3] / [openmpi]
>Using MPI Details [open mpi] with MPI Install [openmpi]
> >>> Using group_reports
> Can't use string ("2") as an ARRAY ref while "strict refs" in use at 
> /hpc/home/USERS/mttuserqa/work/svn/ompi/mtt/trunk/lib/MTT/Values.pm line 107.
> 
> Do you have any ideas what it may be?
> 
> P.S. mini.ini is attached.
> _______
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


-- 
Jeff Squyres
jsquy...@cisco.com

[MTT devel] Trac changes for MPI Forum members

[Jeff Squyres]

The Indiana U. admins have consolidated the authentication system for  
SVN and Trac amongst all the following projects:


- hwloc
- PLPA
- MTT
- Open MPI
- OTPO
- MPI Forum

This means that if you're a member of one or more of those projects,  
your SVN and Trac usernames for those projects are shared.  For  
example, if you change your password via the Trac of one of those  
projects, it's changed for all of them.


--
Jeff Squyres
jsquy...@cisco.com

Re: [MTT devel] MTToGDS

[Jeff Squyres]

Mike --

Many thanks!  This rocks.

I'm embarrassed to say that I broke Cisco's MTT a little while ago and  
haven't found the cycles yet to fix it.  This is excellent motivation  
for me to a) fix my MTT runs, and b) start trying to submit to Google.


Woo hoo!


On Sep 29, 2009, at 3:21 PM, Mike Dubman wrote:


Hello guys and gals,

We have completed development and testing of Google DataStore  
support in MTT and are glad to submit it for community tests.




New Files:



The following new files were added to support GDS inside MTT:

1. client/bquery.pl

   Perl-based GDS client, provides basic DB querying/fetching  
capabilities. It creates resultset (files in YAML format) from user- 
provided sql-like query


2. client/breport.pl

   Perl-based report tool, creates excel reports from yaml files,  
generated by bquery.pl tool.


3. client/custom_launchers/

   For brave only: custom launchers for non-standard HPC, mpi-based  
applications


4. lib/MTT/Reporter/MTTGDS.pm

   GDS Reporter, saves mtt results to GDS (see samples/gds-demo.ini  
for configuration examples)


5. lib/MTT/Utils/ClusterInfo.pm

   Helper library to gather node hw/sw configuration information  
which is saved in GDS together with tests results.


6. New TestResults analyzers for HPC applications:

   lib/MTT/test/Analyze/Performance/Fluent.pm

   lib/MTT/test/Analyze/Performance/HPCC.pm

   lib/MTT/test/Analyze/Performance/HPLGDS.pm

   lib/MTT/test/Analyze/Performance/OpenFoam.pm

   lib/MTT/test/Analyze/Performance/PamCrash.pm

7. samples/gds-demo.ini

   Example of howto configure GDS in MTT and run bquery/breport  
tools at the end of MTT session


8. server/gds/

   GDS backend part, which is running at Google and providing Object  
to YAML, YAML to Object translation service as well as helper code  
for bquery.pl DB client.


9. docs/gds/

   Various documentation



Known Issues and Limitations:

==

* lib/MTT/Utils/ClusterInfo.pm uses "sudo" command to gather node`s  
hardware information.


* When using client/custom_launchers/ to run tests, it is impossible  
to kill the test application when timeout reached.




How to start using MTToGDS:

==

* Contact Jeff to provide you with GDS login/password which is  
needed for querying/saving to DB (http://open-mpi-mtt.appspot.com)


* See samples/gds-demo.ini for configuration examples as well as for  
DB querying and reports generation.


* Read Google GQL syntax documentation and use it with bquery.pl in  
order to query objects from GDB.


* The following perl modules are required for all MTToGDS components:
 libYAML
YAML::Syck
YAML::XS

for breport:
GD::Graph
Spreadsheet::WriteExcel

You can install it on linux systems with yum as following:
yum install perl-libyaml perl-YAML-Syck perl-YAML-XS perl-GD-Graph  
perl-Spreadsheet-WriteExcel


Special Thanks to:

==

Igor Ivanov, Andrew Senin, Alexander Alekhin from Argus-Cv.com for  
they contribution in developing and testing of this feature!


Regards

Mike

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
jsquy...@cisco.com

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Jeff Squyres]

On Sep 28, 2009, at 4:20 PM, Ethan Mallove wrote:


I was commenting on both r1304 and r1319.

These INI params:

  on_start
  on_stop

are very similar to these INI params:

  after_each_exec
  before_any_exec
  after_all_exec

My thought was that it would make sense for them to use a similar
naming scheme and implementation (e.g., use suffix "_exec" and be
passed to DoCommand::Cmd()).



Agreed.

--
Jeff Squyres
jsquy...@cisco.com

Re: [MTT devel] [MTT svn] svn:mtt-svn r1319

[Jeff Squyres]

I'm not sure what you mean -- I thought they added a funclect, not a  
field...?


On Sep 24, 2009, at 3:09 PM, Ethan Mallove wrote:


I think on_stop should conform more to these params:

  after_each_exec
  before_any_exec
  after_all_exec

E.g.,

  before_mtt_start_exec
  after_mtt_start_exec

Then have &_process_get_value_option() call DoCommand. Note, DoCommand
is aware of hashbangs (see &_contains_shell_script_characters()), so
_script() might be redundant.

-Ethan


On Thu, Sep/24/2009 07:46:40PM, Mike Dubman wrote:
>Hey Jeff,
>
>On Thu, Sep 24, 2009 at 4:02 PM, Jeff Squyres  
<jsquy...@cisco.com> wrote:

>
>  The DoCommand.pm sub added ":\n" to the beginning to force  
the Bourne

>  shell interpreter. *This is necessary for some cases where an
>  interpreter is not otherwise specified.
>
>Im not familiar with :\n semantics, how does it force Bourne  
shell and

>what it actually does :)? (seems like leftovers from 1960)
>I think when interpreter is not explicitly specified - the  
default user`s

>shell is used.
>see in the DoCommand::Cmd() . it check the buffer`s* first  
line for
>#!/... semantic and if found - saves buffer to file, adds +x  
perm,* and

>just executes it as a regular script.
>
>When I passed a buffer with shell commands but 1st line was not  
#!/bin/sh

>- it* failed with syntax errors.
>
>*
>
>  I see why you did it -- you want the ability to add your own  
interpreter
>  in _script(). *Why not either make a parameter to add  
the ":\n" or
>  not, or better yet, have DoCommand.pm analyze the beginning  
of the
>  string and if it contains "^:\n" or "^#!", then don't add  
anything. *But
>  if it doesn't contain either of those, then prefix it with ": 
\n".*

>
>  How does that sound?
>
>sounds good!
>
>  Also, is "_script()" a good name? *If you can specify  
your own
>  interpreter, it might not be a shell script. *How about  
()?

>
>ok - () it will be!
>*
>
>regards
>
>Mike
>
>  On Sep 24, 2009, at 8:06 AM, mi...@osl.iu.edu wrote:
>
>Author: miked
>Date: 2009-09-24 08:06:04 EDT (Thu, 24 Sep 2009)
>New Revision: 1319
>URL: https://svn.open-mpi.org/trac/mtt/changeset/1319
>
>Log:
>bug fix: CmdScript() - no need to add extra chars "\n:" to  
the start

>of shell script file
>new funclet: shell_script(section,param)
>
>Text files modified:
>*trunk/lib/MTT/DoCommand.pm * * * *| * * 2 +-
>*trunk/lib/MTT/Values/Functions.pm | * *19 + 
++

>*2 files changed, 20 insertions(+), 1 deletions(-)
>
>Modified: trunk/lib/MTT/DoCommand.pm
> 
= 
= 
= 
= 
= 
= 
= 
= 
==

>--- trunk/lib/MTT/DoCommand.pm *(original)
>+++ trunk/lib/MTT/DoCommand.pm *2009-09-24 08:06:04 EDT  
(Thu, 24 Sep

>2009)
>@@ -797,7 +797,7 @@
>* *$cmds =~ s/\"$//
>* * * *if ($cmds =~ s/^\"//);
>
>- * *print $fh ":\n$cmds\n";
>+ * *print $fh "$cmds\n";
>* *close($fh);
>* *chmod(0700, $filename);
>
>Modified: trunk/lib/MTT/Values/Functions.pm
> 
= 
= 
= 
= 
= 
= 
= 
= 
==

>--- trunk/lib/MTT/Values/Functions.pm * (original)
>+++ trunk/lib/MTT/Values/Functions.pm * 2009-09-24 08:06:04  
EDT (Thu,

>24 Sep 2009)
>@@ -3026,4 +3026,23 @@
>* *return md5_hex($str);
>}
>
>+# Run shell commands as a script, i.e
>+#
>+# [mtt]
>+# myscript=<+# #!/bin/sh
>+# pwd
>+# ls
>+# EOT
>+# on_stop=_script("mtt",myscript)
>+#
>+#
>+
>+sub shell_script {
>+ * * * my ($cmd_section, $cmd_param) = @_;
>    + * * * my $cmd = _ini_val($cmd_section, $cmd_param);
>+ * * * my $x = MTT::DoCommand::CmdScript(1, $cmd);
>+ * * * return $x->{result_stdout};
>+}
>+
>1;
>___
>mtt-svn mailing list
>mtt-...@open-mpi.org
>http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>
>  --
>  Jeff Squyres
>  jsquy...@cisco.com
>
>  ___
>  mtt-devel mailing list
>  m

Re: [MTT devel] OTF testing tool

[Jeff Squyres (jsquyres)]

That sounds yummy - andreas can you help out with how to invoke the otf tool?

-jms
Sent from my PDA.  No type good.

- Original Message -
From: mtt-devel-boun...@open-mpi.org <mtt-devel-boun...@open-mpi.org>
To: Development list for the MPI Testing Tool <mtt-de...@open-mpi.org>
Cc: Andreas Kn?pfer <andreas.knuep...@tu-dresden.de>
Sent: Fri Jul 10 14:27:53 2009
Subject: Re: [MTT devel] OTF testing tool

On Fri, Jul/10/2009 09:51:35AM, Jeff Squyres wrote:
> Ethan - have you seen this?
>
> https://svn.open-mpi.org/trac/ompi/ticket/1967
>
> Do you have any cycles to try to integrate it into MTT?  I was slammed this 
> past week and am out on vacation last week.  But I would very much like to 
> get this into regular MTT testing...

I think there's a simpler way to do this without having to create
another Analyze/Correctness_VampirTrace.pm module. E.g., I have some
vampirtrace stuff in my INI that look like this:

  [MPI details: Open MPI]
  ...
  exec:vampir_trace = _executable() --host _hosts() --prefix 
_prefix() _argv()
  ...

  [Test get: trivial]
  module = Trivial

  [Test build: trivial-VampirTrace]
  test_get = trivial
  module = Trivial

  # Use the VampirTrace wrapper compilers, instead of 
  # the plain vanilla MPI wrappers
  trivial_tests_mpicc  = mpicc-vt
  trivial_tests_mpicxx = mpicxx-vt
  trivial_tests_mpif77 = mpif77-vt
  trivial_tests_mpif90 = mpif90-vt

  [Test run: trivial-VampirTrace]
  test_build = trivial-vampirtrace
  pass = ((_wexitstatus(), 0), _trace_files_exist())

  module = Simple
  specify_module = Simple
  simple_only:tests = _executables(".")
  simple_only_if_exec_exists = 1
  mpi_details_exec = vampir_trace

The above just gets and builds Trivial tests. Then instead of running
them via "mpirun", MTT executes them directly to create the trace
files:

  $ c_hello --host foo,bar --prefix /ompi/installation 

If files with the .events.z or .def.z extensions have been created,
then _trace_files_exist() evaluates to true.

Why don't we create another funclet to run "otfinfo" on the trace
files? I can create it, I just need to know what "otfinfo" does to
confirm that the trace files are good.

-Ethan


>
> -- 
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

[MTT devel] OTF testing tool

[Jeff Squyres]

Ethan - have you seen this?

https://svn.open-mpi.org/trac/ompi/ticket/1967

Do you have any cycles to try to integrate it into MTT?  I was slammed  
this past week and am out on vacation last week.  But I would very  
much like to get this into regular MTT testing...


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] use of ()

[Jeff Squyres]

Ah, ok.  Yes, that is cleaner -- so () is of somewhat limited  
value.


I already have a Cisco.pm file, so adding a new funclet is no biggie.   
Thanks for the idea.



On Jul 6, 2009, at 10:54 AM, Ethan Mallove wrote:


On Mon, Jul/06/2009 10:25:51AM, Jeff Squyres wrote:
> I was just trying to use () in an ini file and ran across an  
annoying

> restriction: I had to make the whole thing be one long line:
>
> max_test_num = <> ('open(IN, "./mpi_test_suite -l|") || die("cant open"); while  
() {
> if (m/Num Tests : (\d+)/) { close(IN); return $1; } } close(IN);  
return

> "0"; ')
> EOF
>
> Without that, the MTT parser would complain that it couldn't find  
the
> closing ' quote (i.e., "('  ...  ')").  I tracked it down and  
it's

> because if I did this:
>
> max_test_num = < ('
> open(IN, "./mpi_test_suite -l|") || die("cant open");
> while () {
> if (m/Num Tests : (\d+)/) { close(IN); return $1; }
> }
> close(IN);
> return "0";
> ')
> EOF
>
> then the parser stops looking for the closing quote on the  
"('" line

> -- it doesn't go beyond the \n.  Yuck.

Ewww. I think the parser should be able to handle those newlines.

>
> Any suggestions?

When I need a multi-line (), I throw the perl code into a sub in
my funclet file. E.g., do this in a Cisco.pm file:

  sub max_test_num {
open(IN, "./mpi_test_suite -l|") || die("cant open");
while () {
if (m/Num Tests : (\d+)/) {
  close(IN);
  return $1;
}
}
close(IN);
return "0";
  }

Put your funclet file next to your INI file, since your INI file will
now require your funclet file:

  foo/bar.ini
  funclets/Cisco.pm

And then have this in your INI file:

  [MTT]
  ...
  funclet_files = ("@INI_NAME@")/../funclets/Cisco.pm
  ...
  max_test_num = ::max_test_num()

-Ethan


>
> --
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel




--
Jeff Squyres
Cisco Systems

[MTT devel] Moving to Mercurial

[Jeff Squyres]

Heads up for all MTT'ers...

This project will eventually be moving away from Subversion and moving  
to Mercurial.  We don't have an exact date when this will happen  
because some infrastructure work needs to occur at Indiana U. first.   
This infrastructure work is currently ongoing; it *may* be ready in a  
week or three from now.


MTT and PLPA are going to be the first two projects (out of the OMPI  
set of projects) to move over to Mercurial.  To be blunt: we're going  
to be the guinnea pigs.  Since MTT and PLPA are both pretty small  
projects, it was estimated that the impact of moving these two  
projects would be small, even if there are problems along the way.


Specifics are TBD, but we'll likely leave MTT's SVN in place for a  
little while after the switch because production sites are using it  
for their nightly regression testing.  But once MTT's official  
development switches to Mercurial, that SVN will become static/read- 
only.  Once the new Mercurial infrastructure stabilizes, we'll  
encourage all MTT users to switch to the Mercurial checkout and  
eventually remove the MTT SVN.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT

[Jeff Squyres]

Shiqing and I talked off-list and fixed the problem.  I committed to  
both trunk and the v3.0 branch.



On Apr 30, 2009, at 4:10 AM, Shiqing Fan wrote:


Hi Jeff,

Yes, I just found out that "-o" option doesn't exist on OS X. But
problem is we don't have client or whatami on Cygwin, are they Linux
shell commands?

Anyway, I figured out another way to get the system type with Perl,  
just
"my $sys_type=$^O" will give us the same result as using "uname -o".  
I'm
not a Perl expert, so I don't know how it works, but here is a link  
from

Perl dev-reference:
http://perl.devquickref.com/perl-get-current-operating-system.html

Could this be a solution?


Regards,
Shiqing


Jeff Squyres wrote:
> Shiqing --
>
> It looks like the "uname -o" that was added into OMPI.pm is
> problematic on OS X.
>
> What does client/whatami/whatami return on the platforms that you  
care

> about?  I.e., can we re-write this chunk of code in
> MPI/Install/OMPI.pm to use whatami output:
>
> my $sys_type=`uname -o`;
> if(($sys_type =~ /cygwin/i || $sys_type =~ /msys/i) &&
> $config->{compiler_name} eq "microsoft") {
> $install = MTT::Common::Cmake::Install($gnu);
> } else {
> $install = MTT::Common::GNU_Install::Install($gnu);
> }
>


--
--
Shiqing Fan  http://www.hlrs.de/people/fan
High Performance Computing   Tel.: +49 711 685 87234
  Center Stuttgart (HLRS)    Fax.: +49 711 685 65832
Address:Allmandring 30   email: f...@hlrs.de
70569 Stuttgart




--
Jeff Squyres
Cisco Systems

[MTT devel] Check a v3.0 commit

[Jeff Squyres]

Can one of you guys sanity https://svn.open-mpi.org/trac/mtt/changeset/1283 
 before I move it to the 3.0 branch?


It should save some testing cycles if the OMPI tarball hasn't changed  
versions from one day to the next (and you start using the field  
ompi_snapshot_version_file in your INI file).


--
Jeff Squyres
Cisco Systems

[MTT devel] MTT SVN branches

[Jeff Squyres]

I have created two new MTT branches:

https://svn.open-mpi.org/svn/mtt/branches/v2.0/ -- the old "ompi- 
core-testers" branch
https://svn.open-mpi.org/svn/mtt/branches/v3.0/ -- a copy from  
the trunk as of yesterday


We are now encouraging everyone to use the v3.0 branch in production.   
New development is going to occur on the trunk that may potentially  
destabilize the trunk at times.  The old "ompi-core-testers" branch  
will be deleted on 25 May 2009 -- anyone still using that branch is  
highly encouraged to move to the new v3.0 branch (and update your INI  
file syntax to match), or move to the v2.0 branch if that is not  
possible.


I have also updated the "how to run OMPI regression with MTT" wiki  
page to point to the /v3.0/ branch:


https://svn.open-mpi.org/trac/mtt/wiki/OMPITesting

--
Jeff Squyres
Cisco Systems

[MTT devel] MTT GAE/GDS call this morning

[Jeff Squyres]

Ok, I opened a Google account with the name "openmpi" this morning,  
and using Ethan's cell phone (heh; turns out that I had already used  
mine), we verified it for use with GAE.  Woo hoo!


"open-mpi-mtt" is the name of the Google Apps application.

I'll send the relevant authentication information out-of-band.

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

I did specifically ask for dancing bears.  ;-)

On Apr 22, 2009, at 10:58 AM, Ethan Mallove wrote:


Dancing bears on slide 1. We're off to a good start.

-Ethan

On Wed, Apr/22/2009 09:11:57AM, Jeff Squyres wrote:
> The slides will also be on webex on the call tomorrow.  Use the  
URL to join
> the meeting in the email invite that you got.  That URL will  
launch an
> application thingy for the web portion of the meeting, and it will  
prompt

> you for a phone number to call you to join the audio portion of the
> meeting.  Mike will be transmitting his slides on the web portion  
(just

> because we can / it's fun ;-) ).
>
>
> On Apr 22, 2009, at 8:39 AM, Mike Dubman wrote:
>
>> Hello guys,
>>
>> Here is a small ppt with MTToGDS summary for tomorrow`s meeting.
>> regards
>>
>> Mike
>>
>> On Thu, Apr 16, 2009 at 5:02 PM, Jeff Squyres  
<jsquy...@cisco.com> wrote:
>> Will there be dancing bears on the slides?  I'll only accept  
slides with

>> dancing bears!
>>
>> ;-)
>>
>> (no need to be formal; if slides help, great, otherwise don't  
make slides

>> just because we have webex available)
>>
>>
>>
>> On Apr 16, 2009, at 9:50 AM, Mike Dubman wrote:
>>
>> I will prepare ppt with summary of what were discussed and agreed,
>> milestones, open questions and other thoughts.
>> regards
>>
>> Mike
>>
>> On Thu, Apr 16, 2009 at 2:07 PM, Jeff Squyres  
<jsquy...@cisco.com> wrote:
>> Ok, I think we converged on a time: 9am US Eastern / 4pm Israel  
next

>> Thuesday, April 23.
>>
>> I'll send the webex invites in a separate email.  Mike: if you  
have slides
>> or other electronic material to show during the call, we can use  
webex for
>> that.  Otherwise, we can just use the telephone part of webex and  
ignore

>> the web part.
>>
>>
>>
>> On Apr 15, 2009, at 4:18 PM, Josh Hursey wrote:
>>
>> I have been listening in on the thread, but have not had time to
>> really look at much (which is why I have not been replying). I'm
>> interested in listening in on the teleconf as well, though if I  
become

>> a blocker for finding a time feel free to cut me out.
>>
>> Best,
>> Josh
>>
>> On Apr 14, 2009, at 8:51 PM, Jeff Squyres wrote:
>>
>> >> BTW -- if it's useful to have a teleconference about this kind  
of
>> >> stuff, I can host a WebEx meeting.  WebEx has local dialins  
around

>> >> the world, including Israel...
>> >>
>> >>
>> >> sure, what about next week?
>> >
>> > I have a Doodle account -- let's try that to do the scheduling:
>> >
>> >http://doodle.com/gzpgaun2ef4szt29
>> >
>> > Ethan, Josh, and I are all in US Eastern timezone (I don't know  
if

>> > Josh will participate), so that might make scheduling *slightly*
>> > easier.  I started timeslots at 8am US Eastern and stopped as  
2pm US
>> > Eastern -- that's already pretty late in Israel.  I also didn't  
list

>> > Friday, since that's the weekend in Israel.
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>>
>> --
>> Jeff Squyres
>> Cisco Systems
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>>
>> --
>> Jeff Squyres
>> Cisco Systems
>>
>> _______
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
>
> --
> Jeff Squyres
> Cisco Systems
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

The slides will also be on webex on the call tomorrow.  Use the URL to  
join the meeting in the email invite that you got.  That URL will  
launch an application thingy for the web portion of the meeting, and  
it will prompt you for a phone number to call you to join the audio  
portion of the meeting.  Mike will be transmitting his slides on the  
web portion (just because we can / it's fun ;-) ).



On Apr 22, 2009, at 8:39 AM, Mike Dubman wrote:


Hello guys,

Here is a small ppt with MTToGDS summary for tomorrow`s meeting.
regards

Mike

On Thu, Apr 16, 2009 at 5:02 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:
Will there be dancing bears on the slides?  I'll only accept slides  
with dancing bears!


;-)

(no need to be formal; if slides help, great, otherwise don't make  
slides just because we have webex available)




On Apr 16, 2009, at 9:50 AM, Mike Dubman wrote:

I will prepare ppt with summary of what were discussed and agreed,  
milestones, open questions and other thoughts.

regards

Mike

On Thu, Apr 16, 2009 at 2:07 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:
Ok, I think we converged on a time: 9am US Eastern / 4pm Israel next  
Thuesday, April 23.


I'll send the webex invites in a separate email.  Mike: if you have  
slides or other electronic material to show during the call, we can  
use webex for that.  Otherwise, we can just use the telephone part  
of webex and ignore the web part.




On Apr 15, 2009, at 4:18 PM, Josh Hursey wrote:

I have been listening in on the thread, but have not had time to
really look at much (which is why I have not been replying). I'm
interested in listening in on the teleconf as well, though if I become
a blocker for finding a time feel free to cut me out.

Best,
Josh

On Apr 14, 2009, at 8:51 PM, Jeff Squyres wrote:

>> BTW -- if it's useful to have a teleconference about this kind of
>> stuff, I can host a WebEx meeting.  WebEx has local dialins around
>> the world, including Israel...
>>
>>
>> sure, what about next week?
>
> I have a Doodle account -- let's try that to do the scheduling:
>
>http://doodle.com/gzpgaun2ef4szt29
>
> Ethan, Josh, and I are all in US Eastern timezone (I don't know if
> Josh will participate), so that might make scheduling *slightly*
> easier.  I started timeslots at 8am US Eastern and stopped as 2pm US
> Eastern -- that's already pretty late in Israel.  I also didn't list
> Friday, since that's the weekend in Israel.

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

Will there be dancing bears on the slides?  I'll only accept slides  
with dancing bears!


;-)

(no need to be formal; if slides help, great, otherwise don't make  
slides just because we have webex available)



On Apr 16, 2009, at 9:50 AM, Mike Dubman wrote:

I will prepare ppt with summary of what were discussed and agreed,  
milestones, open questions and other thoughts.

regards

Mike

On Thu, Apr 16, 2009 at 2:07 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:
Ok, I think we converged on a time: 9am US Eastern / 4pm Israel next  
Thuesday, April 23.


I'll send the webex invites in a separate email.  Mike: if you have  
slides or other electronic material to show during the call, we can  
use webex for that.  Otherwise, we can just use the telephone part  
of webex and ignore the web part.




On Apr 15, 2009, at 4:18 PM, Josh Hursey wrote:

I have been listening in on the thread, but have not had time to
really look at much (which is why I have not been replying). I'm
interested in listening in on the teleconf as well, though if I become
a blocker for finding a time feel free to cut me out.

Best,
Josh

On Apr 14, 2009, at 8:51 PM, Jeff Squyres wrote:

>> BTW -- if it's useful to have a teleconference about this kind of
>> stuff, I can host a WebEx meeting.  WebEx has local dialins around
>> the world, including Israel...
>>
>>
>> sure, what about next week?
>
> I have a Doodle account -- let's try that to do the scheduling:
>
>http://doodle.com/gzpgaun2ef4szt29
>
> Ethan, Josh, and I are all in US Eastern timezone (I don't know if
> Josh will participate), so that might make scheduling *slightly*
> easier.  I started timeslots at 8am US Eastern and stopped as 2pm US
> Eastern -- that's already pretty late in Israel.  I also didn't list
> Friday, since that's the weekend in Israel.

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

Ok, I think we converged on a time: 9am US Eastern / 4pm Israel next  
Thuesday, April 23.


I'll send the webex invites in a separate email.  Mike: if you have  
slides or other electronic material to show during the call, we can  
use webex for that.  Otherwise, we can just use the telephone part of  
webex and ignore the web part.



On Apr 15, 2009, at 4:18 PM, Josh Hursey wrote:


I have been listening in on the thread, but have not had time to
really look at much (which is why I have not been replying). I'm
interested in listening in on the teleconf as well, though if I become
a blocker for finding a time feel free to cut me out.

Best,
Josh

On Apr 14, 2009, at 8:51 PM, Jeff Squyres wrote:

>> BTW -- if it's useful to have a teleconference about this kind of
>> stuff, I can host a WebEx meeting.  WebEx has local dialins around
>> the world, including Israel...
>>
>>
>> sure, what about next week?
>
> I have a Doodle account -- let's try that to do the scheduling:
>
>http://doodle.com/gzpgaun2ef4szt29
>
> Ethan, Josh, and I are all in US Eastern timezone (I don't know if
> Josh will participate), so that might make scheduling *slightly*
> easier.  I started timeslots at 8am US Eastern and stopped as 2pm US
> Eastern -- that's already pretty late in Israel.  I also didn't list
> Friday, since that's the weekend in Israel.

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

On Apr 15, 2009, at 9:14 AM, Mike Dubman wrote:

Hmm.  Ok, so you're saying that we define a "phase object" (for each  
phase) with all the fields that we expect to have, but if we need  
to, we can create fields on the fly, and google will just "do the  
right thing" and associate *all* the data (the "expected" fields and  
the "dynamic" fields) together?


yep. correct. We can define only static attributes (which we know  
for sure should present in every object of given type and leave  
phase specific attributes to stay dynamic)


Hmm.  I would think that even in each phase, we have a bunch of fields  
that we *know* we want to have, right?



I have a Doodle account -- let's try that to do the scheduling:

   http://doodle.com/gzpgaun2ef4szt29

Ethan, Josh, and I are all in US Eastern timezone (I don't know if  
Josh will participate), so that might make scheduling *slightly*  
easier.  I started timeslots at 8am US Eastern and stopped as 2pm US  
Eastern -- that's already pretty late in Israel.  I also didn't list  
Friday, since that's the weekend in Israel.


can we do it on your morining? (our after noon) :)



Visit the Doodle URL (above) and you'll see.  :-)

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

just a local proxy agent that will end up submitting our  
data to $datastore_path, which actually resides at Google?  Do we  
have to use a specific google username/URL to submit (and query)  
results?



You need to download google`s sdk (dev_appserver.py is a part of  
it). In order to develop for gds you  run your code inside sdk  
locally, and when feel comfortable with it - you upload it to the  
google cluster. In order to run attached example, you need to  
download sdk, put it in the following dir hierarchy:


somedir/sdk
somedir/vbench-dev

and run start_datastore.sh, which will run local instance of GDS on  
your machine.Then in another shell you need to run vbech-dev.py,  
which simulates mtt client accessing GDS, storing some objects in  
according to proposed models and then running some sql-like quires  
to fetch and manipulate results.


see 
http://code.google.com/appengine/docs/python/gettingstarted/devenvironment.html


Ah, I see.  Makes sense.

- there's no comments in vbench-dev.py -- can you explain what's  
going on in there?  Can you explain how we would use these scripts?


This is a mtt simulator, it implements google appengine API to  
receive HTTP requests and call appropriate callbacks. (there is a  
map of specific urls to callbacks).


The main callback (which intercepts http GET requests to specific  
URL) runs the test code which creates objects defined in models.py,  
groups many test results into MTTSession and they run some queries  
to access previously created objects.


The real mtt client will use URL pointing to MTT python code running  
at google`s cluster, and use near same code to create/query/ 
manipulate objects defined in models.py.


Ok.  But this code should really be intercepting PUT (or POST)  
requests, not GET, right?


I ask because the MTT client currently POST's the data to send it via  
HTTP to the remote server.


- it *looks* like these scripts are for storing data out in the  
GDS.  Have you looked at the querying side?  Do we know that storing  
data in the form you listed in models.py are easily retrievable in  
the ways that we want?  E.g., can you mock up queries that resemble  
the queries we currently have in our web-based query system today,  
just to show that storing the data in this way will actually allow  
us to do the kinds of queries that we want to do?


I think vbench-dev.py shows some querying capabilities for stored  
objects, there are many ways to query objects by object CLASS and  
Attributes.

see many examples here:

see http://code.google.com/appengine/docs/python/gettingstarted/usingdatastore.html 
 for more querying examples we can use.


Ok.

My only point is that we might want to think a little about the  
queries we want to do when designing the interfaces to stuff all the  
data into the GDS -- it may be helpful to have *some* structure to the  
data that goes into GDS if it helps the queries that we ultimately  
want to do.


Do you want to try making queries for the data that you're shoving  
into GDS that simulate some of the same queries that we can perform  
today?  This will just help validate a) that we can move current  
functionality up to GDS, and b) we can easily make up some new queries  
that we *can't* easily do on postgres today -- it might be fun/useful  
to see if GDS can handle such queries.


Maybe the first goal should be -- once you guys get a good  
understanding of using GDS -- will be to have an MTT Reporter that we  
can all start using to start stuffing data into GDS.  Once we have a  
bit of data out there, you can start trying to query the data and see  
what kinds of capabilities the query side has. Since we have basically  
limitless ability to generate data to submit into GDS :-), if we screw  
up the first few model definitions and end up wiping the data and  
starting over during this development process, it's no big deal --  
just wait one day and the GDS will be populated again with new data  
from our MTT runs.  :-)


What do you think?

In short: I think I'm missing much of the back-story / rationale of  
how the scripts in your tarball work / are to be used.


BTW -- if it's useful to have a teleconference about this kind of  
stuff, I can host a WebEx meeting.  WebEx has local dialins around  
the world, including Israel...



sure, what about next week?


I have a Doodle account -- let's try that to do the scheduling:

http://doodle.com/gzpgaun2ef4szt29

Ethan, Josh, and I are all in US Eastern timezone (I don't know if  
Josh will participate), so that might make scheduling *slightly*  
easier.  I started timeslots at 8am US Eastern and stopped as 2pm US  
Eastern -- that's already pretty late in Israel.  I also didn't list  
Friday, since that's the weekend in Israel.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

- adhere to the same format and the  
reporter can therefore report on it coherently.


For the attachment...

I can "sorta read" python, but I'm not familiar with its intricacies  
and its internal APIs.


- models.py: looks good.  I don't know if *all* the fields we have are  
listed here; it looks fairly short to me.  Did you attempt to include  
all of the fields we submit through the various phases in Reporter are  
there, or did you intentionally leave some out?  (I honestly haven't  
checked; it just "feels short" to me compared to our SQL schema).


--> meta question: is it in the zen of GDS to not have too many index  
fields like you would in SQL?  I.e., if you want to do an operation on  
GDS that you would typically use an SQL index field for, is the idea  
that you would do a map/reduce to select the data instead of an index  
field?


- start_datastore.sh: hmm.  This script seems to imply that the  
datastore is *local*!  Don't we have to HTTP submit the results to  
Google?  More specifically: what is dev_appserver.py?  Is that,  
perchance, just a local proxy agent that will end up submitting our  
data to $datastore_path, which actually resides at Google?  Do we have  
to use a specific google username/URL to submit (and query) results?


- there's no comments in vbench-dev.py -- can you explain what's going  
on in there?  Can you explain how we would use these scripts?


- it *looks* like these scripts are for storing data out in the GDS.   
Have you looked at the querying side?  Do we know that storing data in  
the form you listed in models.py are easily retrievable in the ways  
that we want?  E.g., can you mock up queries that resemble the queries  
we currently have in our web-based query system today, just to show  
that storing the data in this way will actually allow us to do the  
kinds of queries that we want to do?


In short: I think I'm missing much of the back-story / rationale of  
how the scripts in your tarball work / are to be used.


BTW -- if it's useful to have a teleconference about this kind of  
stuff, I can host a WebEx meeting.  WebEx has local dialins around the  
world, including Israel...


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

On Mar 23, 2009, at 9:05 AM, Ethan Mallove wrote:


  ---+-+--
  Resource   | Unit| Unit cost
  ---+-+--
  Outgoing Bandwidth | gigabytes   | $0.12
  Incoming Bandwidth | gigabytes   | $0.10
  CPU Time   | CPU hours   | $0.10
  Stored Data| gigabytes per month | $0.15
  Recipients Emailed | recipients  | $0.0001
  ---+-+--

Would we itemize the MTT bill on a per user basis?  E.g., orgs that
use MTT more, would have to pay more?




Let's assume stored data == incoming bandwidth, because we never throw  
anything away.  And let's go with the SWAG of 100GB.  We may or may  
not be able to gzip the data uploading to the server.  So if anything,  
we *might* be able to decrease the incoming data and have higher level  
of stored data.


I anticipate our outgoing data to be significantly less, particularly  
if we can gzip the outgoing data (which I think we can).  You're  
right, CPU time is a mystery -- we won't know what it will be until we  
start running some queries to see what happens.


100GB * $0.10 = $10
100GB * $0.15 = $15
total = $25 for the first month

So let's SWAG at $25/mo for a year = $300.  This number will be wrong  
for several reasons, but it at least gives us a ballpark.  For $300/ 
year, I think we (the OMPI project) can find a way to pay for this  
fairly easily.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

Yes, I think you're right -- making a "schema" for the datastore might  
be quite easy.  I'm on travel all this week and likely won't be able  
to look into this stuff -- can you guys post a proposal and we can  
dive into it from that angle?


On Mar 22, 2009, at 6:48 AM, Mike Dubman wrote:


Hello guys,

I`m not sure if we should preserve current DB schema, from one  
simple reason - datastore is an object oriented storage and have  
different rules and techniques then rdbms.
The basic storage unit in the datastore is an object which can be  
saved, loaded and queried.

(hadoop is based on the same principles, but open source.)

It seems that DB model for mtt over datastore should not be complex  
at all. The current mtt db schema is mostly optimized for specific  
queries dictated by web UI. Datastore creates indexes automatically,  
based on submitted queries history.


I suggest we discuss/exchange db layout proposals by emails and when  
we get to some general understanding how it should look like - we  
switch to telepresence.


Also, It seems not problem at all to get datastore access for  
existing gmail account. You get 500MB quota for storage. It takes  
5min to start using it.


Here is some short info for datastore API:
- howto submit data model to datastore
- howto save, load, query

http://code.google.com/appengine/docs/python/gettingstarted/usingdatastore.html

please comment.

Thanks

Mike

On Fri, Mar 20, 2009 at 5:38 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:

On Mar 20, 2009, at 10:42 AM, Josh Hursey wrote:

Yeah I think this sounds like a good way to move forward with this
work. The database schema is pretty complex. If you need help on the
database side of things let me know.

To get started, would it be useful to have a meeting over the phone/
telepresence to design the datastore layout? This gives us an
opportunity to start from a blank slate with regards to the
datastore, so it may be useful brainstorm a bit beforehand.


Yes, it probably would.  My understanding of hadoop (which is very  
highlevel) is that just dump everything in without too much concern  
about the structure / "schema".  But I could be wrong on that.



The Google Apps account is under my personal Google account, so I'm
reluctant to use it. I think the reason it took so long for me, was
because when I originally signed up it was in limited beta. I think
the approval time is much shorter now (maybe a day?), and we can make
an openmpi or mtt account that we can use.

With regard to Hadoop, I don't think that IU has a set of machines
that would work, but I can ask around. We could always try Hadoop on
a single machine if people wanted to play around with data querying/
storage.

I don't have a strong preference either way, but Google Apps may
provide us with a lower overhead solution for the long run even
though it costs $$.



It looks like there is a set that you can use for free.  When you go  
over one of several metrics (CPU hours/day, storage, bandwidth in,  
bandwidth out, etc.), then you have to start paying.  But even with  
that, the costs look *quite* reasonable and should be easily covered  
by the combined Open MPI organizations (I'm talking hundreds of  
dollars here, not tens of thousands).



--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1273(Analyze/Performance plug-ins)

[Jeff Squyres]

t; +package MTT::Test::Analyze::Performance::HPL;
>>> +use strict;
>>> +use Data::Dumper;
>>> +#use MTT::Messages;
>>> +
>>> +# Process the result_stdout emitted from one of hpl tests
>>> +sub Analyze {
>>> +
>>> + * *my($result_stdout) = @_;
>>> + * *my $report;
>>> + * *my(@t_v,
>>> + * * * @time,
>>> + * * * @gflops);
>>> +
>>> +$report->{test_name}="HPL";
>>> + * *my @lines = split(/\n|\r/, $result_stdout);
>>> + * *# Sample result_stdout:
>>> +#- The matrix A is randomly generated for each test.
>>> +#- The following scaled residual check will be computed:
>>> +# * * *||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||
>>> _oo )
>>  * N )
>>> +#- The relative machine precision (eps) is taken to be * * * * *
>>> * *
>>  1.110223e-16
>>> +#- Computational tests pass if scaled residuals are less than *
>>> * * *
>>  * * * *16.0
>>>
>>
>>  
+#===

>> =
>>> +#T/V * * * * * * * *N * *NB * * P * * Q * * * * * * * Time * * *
>>> * *
>>  * * * Gflops
>>>
>>
>>  
+#---

>> -
>>> +#WR00L2L2 * * * 29184 * 128 * * 2 * * 4 * * * * * 15596.86 * * *
>>> * *
>>  * *1.063e+00
>>>
>>
>>  
+#---

>> -
>>> +#||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)= * * *  
*0.0008986

>>  .. PASSED
>>>
>>
>>  
+#===

>> =
>>> +#T/V * * * * * * * *N * *NB * * P * * Q * * * * * * * Time * * *
>>> * *
>>  * * * Gflops
>>>
>>
>>  
+#---

>> -
>>> +#WR00L2L4 * * * 29184 * 128 * * 2 * * 4 * * * * * 15251.81 * * *
>>> * *
>>  * *1.087e+00
>>> + * *my $line;
>>> + * *while (defined($line = shift(@lines))) {
>>> + * * * *#WR00L2L2 * * * 29184 * 128 * * 2 * * 4 * * * * *
>>> 15596.86 *
>>  * * * * * *1.063e+00
>>> + * * * *if ($line =~
>>  m/^(\S+)\s+\d+\s+\d+\s+\d+\s+\d+\s+(\d+[\.\d]+)\s+(\S+)/) {
>>> + * * * * * *push(@t_v, $1);
>>> + * * * * * *push(@time, $2);
>>> + * * * * * *push(@gflops, $3);
>>> + * * * *}
>>> + * *}
>>> +
>>> + * * *# Postgres uses brackets for array insertion
>>> + * *# (see postgresql.org/docs/7.4/interactive/arrays.html)
>>> + * *$report->{tv} * = "{" . join(",", @t_v) . "}";
>>> + * *$report->{time} * = "{" . join(",", @time) . "}";
>>> + * *$report->{gflops} * = "{" . join(",", @gflops) . "}";
>>> + * *return $report;
>>> +}
>>> +
>>> +1;
>>> +
>>> ___
>>> mtt-svn mailing list
>>> mtt-...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>>
>> References
>>
>>Visible links
>>. http://www.netlib.org/benchmark/hpl/
>>. mailto:ethan.mall...@sun.com
>>. mailto:mi...@osl.iu.edu
>>. https://svn.open-mpi.org/trac/mtt/changeset/1273
>>. http://postgresql.org/docs/7.4/interactive/arrays.html
>>. mailto:mtt-...@open-mpi.org
>>. http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn
>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

On Mar 20, 2009, at 10:42 AM, Josh Hursey wrote:


Yeah I think this sounds like a good way to move forward with this
work. The database schema is pretty complex. If you need help on the
database side of things let me know.

To get started, would it be useful to have a meeting over the phone/
telepresence to design the datastore layout? This gives us an
opportunity to start from a blank slate with regards to the
datastore, so it may be useful brainstorm a bit beforehand.



Yes, it probably would.  My understanding of hadoop (which is very  
highlevel) is that just dump everything in without too much concern  
about the structure / "schema".  But I could be wrong on that.



The Google Apps account is under my personal Google account, so I'm
reluctant to use it. I think the reason it took so long for me, was
because when I originally signed up it was in limited beta. I think
the approval time is much shorter now (maybe a day?), and we can make
an openmpi or mtt account that we can use.

With regard to Hadoop, I don't think that IU has a set of machines
that would work, but I can ask around. We could always try Hadoop on
a single machine if people wanted to play around with data querying/
storage.

I don't have a strong preference either way, but Google Apps may
provide us with a lower overhead solution for the long run even
though it costs $$.




It looks like there is a set that you can use for free.  When you go  
over one of several metrics (CPU hours/day, storage, bandwidth in,  
bandwidth out, etc.), then you have to start paying.  But even with  
that, the costs look *quite* reasonable and should be easily covered  
by the combined Open MPI organizations (I'm talking hundreds of  
dollars here, not tens of thousands).


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

Looks like there were 400 applications this year; they selected 150 --  
38%.  We were in the unlucky 62%.  Bummer.


On Mar 18, 2009, at 4:05 PM, Ethan Mallove wrote:


On Wed, Mar/18/2009 03:28:48PM, Josh Hursey wrote:
> So they posted the list of accepted projects and we are -not- on it
> for this year:
>
>   http://socghop.appspot.com/program/accepted_orgs/google/gsoc2009
>
> Maybe next year. I don't know if they will be sending around a note
> regarding why we were not selected to participate. If they do I will
> forward it along.

Thanks, Josh.

I'm reading that in 2008, they only accepted 174 out of the 7100
applications.

-Ethan

>
> Cheers,
> Josh
>
> On Mar 13, 2009, at 3:19 PM, Jeff Squyres wrote:
>
>> Awesome; many thanks for carrying the baton over the finish line,  
Josh!

>>
>> On Mar 13, 2009, at 2:56 PM, Josh Hursey wrote:
>>
>>> The application has been submitted. We find out on March 18 (3  
pm) if

>>> we have been accepted. Link to timeline below:
>>>http://socghop.appspot.com/document/show/program/google/gsoc2009/
>>> timeline
>>>
>>> Cheers,
>>> Josh
>>>
>>> On Mar 13, 2009, at 2:19 PM, Josh Hursey wrote:
>>>
>>> > I just pushed a final draft to the repository. I'll probably  
plan

>>> > on submitting at 2:30/2:45. Let me know if you have any edits
>>> > before then either through email or IM.
>>> >
>>> > Cheers,
>>> > Josh
>>> >
>>> > On Mar 13, 2009, at 12:11 PM, Josh Hursey wrote:
>>> >
>>> >> I finished a first pass at cleaning up the Ideas page on the  
Wiki.
>>> >> All of the ideas were preserved, just some rewording and  
formatting.

>>> >>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>> >>
>>> >> If you get a chance, read through this and make sure the text
>>> >> sounds ok (feel free to clean the text up as necessary).
>>> >>
>>> >> The application is due by 3 pm EST. So I hope to have the
>>> >> application ready by 2ish. I'll move onto the application  
itself now.

>>> >>
>>> >> -- Josh
>>> >>
>>> >> On Mar 12, 2009, at 4:43 PM, Josh Hursey wrote:
>>> >>
>>> >>> Jeff is going to take the first pass at the application.
>>> >>>
>>> >>> I am going to go through the Idea page on the wiki and  
polish a bit:

>>> >>>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>> >>>
>>> >>> I'll let folks know when I'm done, and we can start  
iterating on

>>> >>> drafts.
>>> >>>
>>> >>> Cheers,
>>> >>> Josh
>>> >>>
>>> >>> On Mar 12, 2009, at 4:08 PM, Jeff Squyres wrote:
>>> >>>
>>> >>>> I've created a quick-n-dirty hg to collaborate on the GSOC
>>> >>>> application.  There's a web form to fill out to apply, so  
let's

>>> >>>> work on a .txt file in the hg to get it right.
>>> >>>>
>>> >>>> We have until 3pm US Eastern time tomorrow to submit.  Here's
>>> >>>> the HG:
>>> >>>>
>>> >>>>ssh://www.open-mpi.org/~jsquyres/hg/gsoc/
>>> >>>>
>>> >>>> I've put the PDF there for now; I'll kruft up a quick .txt
>>> >>>> shortly and push it there as well.
>>> >>>>
>>> >>>> --
>>> >>>> Jeff Squyres
>>> >>>> Cisco Systems
>>> >>>>
>>> >>>> ___
>>> >>>> mtt-devel mailing list
>>> >>>> mtt-de...@open-mpi.org
>>> >>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>> >>>
>>> >>> ___
>>> >>> mtt-devel mailing list
>>> >>> mtt-de...@open-mpi.org
>>> >>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>> >>
>>> >> ___
>>> >> mtt-devel mailing list
>>> >> mtt-de...@open-mpi.org
>>> >> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>> >
>>> > ___
>>> > mtt-devel mailing list
>>> > mtt-de...@open-mpi.org
>>> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>>
>>> ___
>>> mtt-devel mailing list
>>> mtt-de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>>
>> --
>> Jeff Squyres
>> Cisco Systems
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt text report oddity

[Jeff Squyres]

On Mar 19, 2009, at 3:19 AM, Mike Dubman wrote:

because the results are rendered in chunks during reporting phase.  
(100 pieces every flush)
This caused same benchmark line to appear more then once in the  
final report.


Ahhh... right.

You can configure the reporter to issue results not by number, but  
for same benchmark at once:


put this in the ini file:

[MTT]
submit_group_results=1


Perfect.  Thanks!

Also, html report is nicer and allows you easy navigation to the  
errors


True, but my HTML files kept getting overwritten by successive submits  
in this case.


--
Jeff Squyres
Cisco Systems

[MTT devel] mtt text report oddity

[Jeff Squyres]

I got a fairly odd mtt text report (it's super wide, sorry):

| Test Run| intel | 1.3.1rc5| 00:12| 5 
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:59| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:08| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:51| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:59| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:48| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:10| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:05| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:09| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:25| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:46| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:59| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:23| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:50| 100  |  
1|  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:56| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:53| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:22| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 04:21| 100   
|  | 1|  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 04:12| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:36| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:48| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:47| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 03:08| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:57| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:43| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|
| Test Run| intel | 1.3.1rc5| 02:48| 101   
|  |  |  | Test_Run-intel- 
developer-1.3.1rc5.html|


Notice that there are *many* "intel" lines, each with 101 passes.  The  
only difference between them is the times that they ran -- but there's  
even repeats of that.


Do we know why there is so many different lines for the intel test  
suite?


Did this get changed in the text reporter changes from Voltaire  
(somewhat) recently?


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

Awesome; many thanks for carrying the baton over the finish line, Josh!

On Mar 13, 2009, at 2:56 PM, Josh Hursey wrote:


The application has been submitted. We find out on March 18 (3 pm) if
we have been accepted. Link to timeline below:
   http://socghop.appspot.com/document/show/program/google/gsoc2009/
timeline

Cheers,
Josh

On Mar 13, 2009, at 2:19 PM, Josh Hursey wrote:

> I just pushed a final draft to the repository. I'll probably plan
> on submitting at 2:30/2:45. Let me know if you have any edits
> before then either through email or IM.
>
> Cheers,
> Josh
>
> On Mar 13, 2009, at 12:11 PM, Josh Hursey wrote:
>
>> I finished a first pass at cleaning up the Ideas page on the Wiki.
>> All of the ideas were preserved, just some rewording and  
formatting.

>>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>
>> If you get a chance, read through this and make sure the text
>> sounds ok (feel free to clean the text up as necessary).
>>
>> The application is due by 3 pm EST. So I hope to have the
>> application ready by 2ish. I'll move onto the application itself  
now.

>>
>> -- Josh
>>
>> On Mar 12, 2009, at 4:43 PM, Josh Hursey wrote:
>>
>>> Jeff is going to take the first pass at the application.
>>>
>>> I am going to go through the Idea page on the wiki and polish a  
bit:

>>>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>>
>>> I'll let folks know when I'm done, and we can start iterating on
>>> drafts.
>>>
>>> Cheers,
>>> Josh
>>>
>>> On Mar 12, 2009, at 4:08 PM, Jeff Squyres wrote:
>>>
>>>> I've created a quick-n-dirty hg to collaborate on the GSOC
>>>> application.  There's a web form to fill out to apply, so let's
>>>> work on a .txt file in the hg to get it right.
>>>>
>>>> We have until 3pm US Eastern time tomorrow to submit.  Here's
>>>> the HG:
>>>>
>>>>ssh://www.open-mpi.org/~jsquyres/hg/gsoc/
>>>>
>>>> I've put the PDF there for now; I'll kruft up a quick .txt
>>>> shortly and push it there as well.
>>>>
>>>> --
>>>> Jeff Squyres
>>>> Cisco Systems
>>>>
>>>> ___
>>>> mtt-devel mailing list
>>>> mtt-de...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>>
>>> ___
>>> mtt-devel mailing list
>>> mtt-de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] GSOC application

[Jeff Squyres]

You have to "module load osl merurial" in your shell startup files  
somewhere for hg to work on milliways.


On Mar 13, 2009, at 3:15 PM, Ethan Mallove wrote:


On Fri, Mar/13/2009 02:19:24PM, Josh Hursey wrote:
> I just pushed a final draft to the repository. I'll probably plan on
> submitting at 2:30/2:45. Let me know if you have any edits before  
then

> either through email or IM.

Where's the repo? I'm looking at the .txt file in ~jsquyres/hg/gsoc,
but it looks incomplete. Does milliways have "hg" somewhere? (I'm
trying to find the parent repo using "hg out".)

-Ethan

>
> Cheers,
> Josh
>
> On Mar 13, 2009, at 12:11 PM, Josh Hursey wrote:
>
>> I finished a first pass at cleaning up the Ideas page on the  
Wiki. All of

>> the ideas were preserved, just some rewording and formatting.
>>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>
>> If you get a chance, read through this and make sure the text  
sounds ok

>> (feel free to clean the text up as necessary).
>>
>> The application is due by 3 pm EST. So I hope to have the  
application

>> ready by 2ish. I'll move onto the application itself now.
>>
>> -- Josh
>>
>> On Mar 12, 2009, at 4:43 PM, Josh Hursey wrote:
>>
>>> Jeff is going to take the first pass at the application.
>>>
>>> I am going to go through the Idea page on the wiki and polish a  
bit:

>>>   https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas
>>>
>>> I'll let folks know when I'm done, and we can start iterating on  
drafts.

>>>
>>> Cheers,
>>> Josh
>>>
>>> On Mar 12, 2009, at 4:08 PM, Jeff Squyres wrote:
>>>
>>>> I've created a quick-n-dirty hg to collaborate on the GSOC  
application.
>>>> There's a web form to fill out to apply, so let's work on  
a .txt file in

>>>> the hg to get it right.
>>>>
>>>> We have until 3pm US Eastern time tomorrow to submit.  Here's  
the HG:

>>>>
>>>>ssh://www.open-mpi.org/~jsquyres/hg/gsoc/
>>>>
>>>> I've put the PDF there for now; I'll kruft up a quick .txt  
shortly and

>>>> push it there as well.
>>>>
>>>> --
>>>> Jeff Squyres
>>>> Cisco Systems
>>>>
>>>> ___
>>>> mtt-devel mailing list
>>>> mtt-de...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>>
>>> ___
>>> mtt-devel mailing list
>>> mtt-de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>>
>> ___
>> mtt-devel mailing list
>> mtt-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

[MTT devel] GSOC application

[Jeff Squyres]

I've created a quick-n-dirty hg to collaborate on the GSOC  
application.  There's a web form to fill out to apply, so let's work  
on a .txt file in the hg to get it right.


We have until 3pm US Eastern time tomorrow to submit.  Here's the HG:

ssh://www.open-mpi.org/~jsquyres/hg/gsoc/

I've put the PDF there for now; I'll kruft up a quick .txt shortly and  
push it there as well.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

Applied.

https://svn.open-mpi.org/trac/mtt/changeset/1272

You should probably push your patch to Filesys::Diskfree upstream to  
the CPAN author:


http://search.cpan.org/~abarclay/Filesys-DiskFree-0.06/DiskFree.pm

(i.e., we wholly imported this module into MTT; we didn't write it)


On Mar 12, 2009, at 9:24 AM, Shiqing Fan wrote:


Hi Jeff,

No problem. I made a tarball for you. :-)


Thanks,
Shiqing

Jeff Squyres wrote:
> This could be an artifact of the new SVN manager system that IU
> installed yesterday; I've sent a mail to the admin asking about it.
> Sorry!  :-(
>
> If you could send me your patch as an attachment, I can commit it  
(OS
> X Mail.app does weird things with the spacing of inlined patches :- 
( ).

>
>
> On Mar 12, 2009, at 7:00 AM, Shiqing Fan wrote:
>
>> Here is some output of 'svn info':
>>
>> Path: .
>> URL: https://svn.open-mpi.org/svn/mtt/trunk
>> Repository Root: https://svn.open-mpi.org/svn/mtt
>> Repository UUID: 3a00f1f0-e206-0410-aee5-9638c71ae14b
>> Revision: 1271
>> Node Kind: directory
>> Schedule: normal
>> Last Changed Author: emallove
>> Last Changed Rev: 1271
>> Last Changed Date: 2009-02-05 18:22:38 +0100 (Thu, 05 Feb 2009)
>>
>>
>> Is that ok, if someone else commits the patch instead of me? I  
don't

>> mind actually. :-)
>>
>>
>> Thanks,
>> Shiqing





--
Jeff Squyres
Cisco Systems

[MTT devel] 1.3.1?

[Jeff Squyres]

So -- RM's -- can we release 1.3.1?  The tarball is ready (it's made  
at the same time as RC tarballs to guarantee that it's the same).  All  
that's necessary is posting it to the web site and sending out the  
announcement.


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

Did you check out via https?  We only allow commits to all OMPI SVN  
repositories via https.


On Mar 12, 2009, at 5:10 AM, Shiqing Fan wrote:


Hi Ethan and Jeff,

Thanks for all your help. Now it's been fixed and tested.

But it seems that I don't have permission to commit the patch ( I got
403 forbidden ). Any idea?


Thanks,
Shiqing





Ethan Mallove wrote:
> On Wed, Mar/11/2009 03:47:22PM, Jeff Squyres wrote:
>
>> Thanks for your patience!  Yes, this looks good to me with one  
minor nit:

>>
>>
>>> +if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
>>> +$config->{compiler_name} == "microsoft") {
>>>
>> should be
>>
>>
>>> +if(($sys_type eq "Cygwin" || $sys_type eq "Msys") &&
>>> +$config->{compiler_name} eq "microsoft") {
>>>
>> Josh / Ethan -- got any comments for Shiqing?
>>
>
> I just noticed last night's CYGWIN results. Nice!
>
> Should a regexp (see below) be used in case the sys_type comes  
back as, e.g.,

> "cygwin", "CYGWIN", "cygwin-2.0", etc.?
>
>   $sys_type =~ /cygwin/i
>
> Other than that, it looks good to me.
>
> -Ethan
>
>
>>
>>
>> On Mar 11, 2009, at 2:58 PM, Shiqing Fan wrote:
>>
>>
>>> Hi Jeff,
>>>
>>>
>>> Sorry for the carelessness. This time it looks better? :-)
>>>
>>>
>>> Thanks,
>>> Shiqing
>>>
>>>> Can you double check your cr / cr/lf settings?  It looks like  
you're
>>>> committing in the windows format -- can you convert and commit  
in the

>>>> unix format?  That way, it wouldn't look like the entire
>>>> GNU_Install.pm file is being replaced, for example.
>>>>
>>>> I forgot to mention in my last mail -- since cmake *only* works  
on
>>>> Windows and our Autotools system *doesn't* work on Windows, I  
retract
>>>> my earlier statement: don't bother with an ompi_cmake parameter  
in the
>>>> .ini file.  Just have OMPI.pm figure out if you're on cygwin or  
mingw

>>>> and call the right back-end building function.
>>>>
>>>> Also, in Do_Step.pm, you might want to remove the "_" prefix  
from the

>>>> sub names.  "_" as a prefix in perl is meant to imply that it's a
>>>> private variable/method.
>>>>
>>>> Finally, in Do_Step.pm, do you have two _do_step() functions?
>>>>
>>>>
>>> Index: lib/Filesys/DiskFree.pm
>>>  
===

>>> --- lib/Filesys/DiskFree.pm (revision 1271)
>>> +++ lib/Filesys/DiskFree.pm (working copy)
>>> @@ -29,6 +29,16 @@
>>> 'inodes' => "df -Pi",
>>> 'format' => "linuxish",
>>> },
>>> +'cygwin' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> +'msys' => {
>>> +   'blocks' => "df -P",
>>> +   'inodes' => "df -Pi",
>>> +   'format' => "linuxish",
>>> +},
>>> 'solaris' =>  {
>>> 'blocks' => "df -k",
>>> 'inodes' => "df -k -o i -F ufs",
>>> Index: lib/MTT/Common/Cmake.pm
>>>  
===

>>> --- lib/MTT/Common/Cmake.pm (revision 0)
>>> +++ lib/MTT/Common/Cmake.pm (revision 0)
>>> @@ -0,0 +1,77 @@
>>> +#!/usr/bin/env perl
>>> +#
>>> +# Copyright (c) 2005-2006 The Trustees of Indiana University.
>>> +# All rights reserved.
>>> +# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights  
reserved.
>>> +# Copyright (c) 2007-2008 Sun Microsystems, Inc.  All rights  
reserved.
>>> +# Copyright (c) 2009  High Performance Computing Center  
Stuttgart,
>>> +# University of Stuttgart.  All rights  
reserved.

>>> +# $COPYRIGHT$
>>> +#
>>> +# Additional copyrights may follow
>>> +#
>>> +# $HEADER$
>>> +#
>>> +
>>> +package MTT::Common::Cmake;
>>> +my $package = ModuleName(__PACKAGE__);
>>> +
>>> +use strict;
>>> +use MTT::Messages;
>>> +use MTT::Values;
>>> +use MTT::Common::Do_step;
>>> +
>>>  
+ 
#---

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

--- $cmd $config- 
>{$arguments_key} result_stdout";

-$result_stdout .= "/result_stderr"
-if ($mss);
-$result_stdout .= " ---\n$ret->{result_stdout}";
-}
-
-# Add header line to stderr
-if (!$mss && defined($ret->{result_stderr}) &&
-$ret->{result_stderr} !~ /^\s*$/) {
-$result_stderr = "--- $cmd $config- 
>{$arguments_key} result_stderr ---\n$ret->{result_stderr}";

-}
-
-# Repeat *only* if $restart_on_pattern is defined
-} while (!MTT::DoCommand::wsuccess($ret->{exit_status}) and
- (defined($restart_on_pattern) &&
-  ($ret->{result_stderr} =~ /$restart_on_pattern/i or
-   $ret->{result_stdout} =~ /$restart_on_pattern/i)  
and

-  $restart_attempts++ < $restart_attempts_max));
-
-# If fail, save the results in {result_stdout} and
-# {result_stderr}.
-if (!MTT::DoCommand::wsuccess($ret->{exit_status})) {
-$ret->{result_message} = "\"$cmd $config- 
>{$arguments_key}\" failed -- skipping this build.";

-# Put the output of the failure into $ret so that it gets
-# reported
-$ret->{result_stdout} = $result_stdout
-if (defined($result_stdout));
-$ret->{result_stderr} = $result_stderr
-if (!$mss && defined($result_stderr));
-$ret->{exit_status} = $ret->{exit_status};
-$ret->{fail} = 1;
-}
-
-# If succeed, keep the stdout/stderr in their respective hash
-# keys for this step.
-else {
-if (defined($result_stdout)) {
-delete $ret->{result_stdout};
-$ret->{$stdout_key} = $result_stdout;
-}
-if (!$mss && defined($result_stderr)) {
-delete $ret->{result_stderr};
-$ret->{$stderr_key} = $result_stderr;
-}
-}
-} else {
-Debug("Skippping '$cmd' step.\n");
-}
-
-if (defined($config->{$after_cmd_key})) {
-_run_step($config->{$after_cmd_key}, $after_cmd_key);
-}
-
-return $ret;
-}
-
-sub _run_step {
-my ($cmd, $step) = @_;
-return MTT::DoCommand::RunStep(1, $cmd, 30, undef, undef, $step);
-}
-
1;
Index: lib/MTT/Defaults.pm
===
--- lib/MTT/Defaults.pm (revision 1271)
+++ lib/MTT/Defaults.pm (working copy)
@@ -3,6 +3,8 @@
# Copyright (c) 2005-2006 The Trustees of Indiana University.
# All rights reserved.
# Copyright (c) 2006-2007 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2009  High Performance Computing Center  
Stuttgart,
+# University of Stuttgart.  All rights  
reserved.

# $COPYRIGHT$
#
# Additional copyrights may follow
@@ -39,7 +41,7 @@
},

known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai",  
"absoft",
-  "pathscale", "sun", "none",  
"unknown" ],
+  "pathscale", "sun", "microsoft",  
"none", "unknown" ],
known_resource_manager_names => [ "slurm", "tm", "loadleveler",  
"n1ge",

  "alps", "none", "unknown" ],
known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx",  
"verbs",

Index: lib/MTT/MPI/Install/OMPI.pm
===
--- lib/MTT/MPI/Install/OMPI.pm (revision 1271)
+++ lib/MTT/MPI/Install/OMPI.pm (working copy)
@@ -3,6 +3,8 @@
# Copyright (c) 2005-2006 The Trustees of Indiana University.
# All rights reserved.
# Copyright (c) 2006-2008 Cisco Systems, Inc.  All rights reserved.
+# Copyright (c) 2009  High Performance Computing Center  
Stuttgart,
+# University of Stuttgart.  All rights  
reserved.

# $COPYRIGHT$
#
# Additional copyrights may follow
@@ -20,6 +22,7 @@
use MTT::Values;
use MTT::Files;
use MTT::Common::GNU_Install;
+use MTT::Common::Cmake;
use MTT::Values::Functions::MPI::OMPI;

#--
@@ -86,7 +89,16 @@
stderr_save_lines => $config->{stderr_save_lines},
merge_stdout_stderr => $config->{merge_stdout_stderr},
};
-my $install = MTT::Common::GNU_Install::Install($gnu);
+
+my $install;
+my $sys_type=`uname -o`;
+if(($sys_type == "Cygwin" || $sys_type == "Msys") &&
+$config->{compiler_name} == "microsoft") {
+$install = MTT::Common::Cmake::Install($gnu);
+} else {
+$install = MTT::Common::GNU_Install::Install($gnu);
+}
+
foreach my $k (keys(%{$install})) {
$ret->{$k} = $install->{$k};
}



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

prefix;
Index: lib/MTT/Defaults.pm
===
--- lib/MTT/Defaults.pm (revision 1271)
+++ lib/MTT/Defaults.pm (working copy)
@@ -39,7 +39,7 @@
},

known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai",  
"absoft",
-  "pathscale", "sun", "none",  
"unknown" ],
+  "pathscale", "sun", "microsoft",  
"none", "unknown" ],
known_resource_manager_names => [ "slurm", "tm", "loadleveler",  
"n1ge",

  "alps", "none", "unknown" ],
known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx",  
"verbs",

Index: lib/MTT/MPI/Install/OMPI.pm
===
--- lib/MTT/MPI/Install/OMPI.pm (revision 1271)
+++ lib/MTT/MPI/Install/OMPI.pm (working copy)
@@ -76,6 +76,7 @@
my $gnu = {
configdir => $config->{configdir},
configure_arguments => $config->{configure_arguments},
+compiler_name => $config->{compiler_name},
vpath => "no",
installdir => $config->{installdir},
bindir => $config->{bindir},



All this looks good. ^^

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT on Windows

[Jeff Squyres]

(moving to mtt-devel)

Shiqing --

In looking at your patch, I see a few nits.  See below.


On Feb 26, 2009, at 12:56 PM, Shiqing Fan wrote:


Index: lib/Filesys/DiskFree.pm

===
--- lib/Filesys/DiskFree.pm (revision 1271)
+++ lib/Filesys/DiskFree.pm (working copy)
@@ -29,6 +29,11 @@
'inodes' => "df -Pi",
'format' => "linuxish",
},
+'cygwin' => {
+   'blocks' => "df -P",
+   'inodes' => "df -Pi",
+   'format' => "linuxish",
+},
'solaris' =>  {
'blocks' => "df -k",
'inodes' => "df -k -o i -F ufs",
Index: lib/MTT/Common/GNU_Install.pm
===
--- lib/MTT/Common/GNU_Install.pm   (revision 1271)
+++ lib/MTT/Common/GNU_Install.pm   (working copy)
@@ -43,7 +43,45 @@
my $ret;
$ret->{test_result} = MTT::Values::FAIL;
$ret->{exit_status} = 0;
+
+if (`uname -o` == "Cygwin") {


Can we think of a better test here?  Will we always be running MTT  
under Cygwin?


Should this code rather be in a different .pm and the user selects  
which one to use via the .ini file?




+# On windows, do the following steps:
+#   [ ] cmake -G "generator" -D configure_arguments  
source_path

+#   [ ] devenv OpenMPI.sln /build

+# generate Visual Studio solution files
+$x = _do_step($config,
+  "cmake $config->{configure_arguments} -D  
CMAKE_INSTALL_PREFIX:PATH=$config->{installdir} .", 1);

+
+# Overlapping keys in $x override $ret
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+if (exists($ENV{LD_LIBRARY_PATH})) {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}: 
$ENV{LD_LIBRARY_PATH}";

+} else {
+$ENV{LD_LIBRARY_PATH} = "$config->{libdir}";
+}
+
+# compile the whole solution
+$x = _do_step($config, "devenv.com OpenMPI.sln /build  
debug", 1);

+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+# install, not working yet


What does this comment mean? ^^^



+$x = _do_step($config, "devenv INSTALL.vcproj /build", 1);
+%$ret = (%$ret, %$x);
+return $ret if (!MTT::DoCommand::wsuccess($ret- 
>{exit_status}));

+
+# All done!
+$ret->{test_result} = MTT::Values::PASS;
+$ret->{result_message} = "Success";
+Debug("Build was a success\n");
+
+return $ret;
+
+}
+
# If the user does not use --prefix on their own, default
# to $installdir
my $prefix;
@@ -224,5 +262,3 @@
my ($cmd, $step) = @_;
return MTT::DoCommand::RunStep(1, $cmd, 30, undef, undef, $step);
}
-
-1;


Don't remove the "1;" at the end of the file...



Index: lib/MTT/Defaults.pm
===
--- lib/MTT/Defaults.pm (revision 1271)
+++ lib/MTT/Defaults.pm (working copy)
@@ -39,7 +39,7 @@
},

known_compiler_names => [ "gnu", "pgi", "ibm", "intel", "kai",  
"absoft",
-  "pathscale", "sun", "none",  
"unknown" ],
+  "pathscale", "sun", "microsoft",  
"none", "unknown" ],
known_resource_manager_names => [ "slurm", "tm", "loadleveler",  
"n1ge",

  "alps", "none", "unknown" ],
known_network_names => [ "tcp", "udp", "ethernet", "gm", "mx",  
"verbs",



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] visual reports for mtt

[Jeff Squyres]

I've added my own ideas and organization to the wiki page.

Ethan / Josh -- want to add anything?

We should work offline on the GSoC application.


On Feb 27, 2009, at 7:59 AM, Mike Dubman wrote:


Hello guys,

Using MapReduce or any other in-memory DB techque sounds cool and  
should provide fast access to all perfomance data.

Here is a wiki page with some ideas for mtt addons: 
https://svn.open-mpi.org/trac/mtt/wiki/MttNewFeaturesIdeas


regards

Mike

On Wed, Feb 25, 2009 at 6:47 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:

Just to bring the info to the list...

There have been some off list e-mails and some in-person discussions  
about a fascinating idea that looks promising here.


How about moving the MTT data store to the Google Apps Engine/ 
datastore?  See here:


   http://code.google.com/appengine/docs/whatisgoogleappengine.html

Josh looked at this about a year ago and thought that there would be  
some interesting possibilities here, but never had the cycles to  
follow up on it.  Specifically: if we move all the MTT data from a  
postgres DB to the Google Apps datastore, we might have a highly  
scalable mechanism for queries, and therefore be able to do much  
more interesting kinds of queries (right now, we're fairly limited  
in our queries because of memory and CPU restrictions via Apache/PHP/ 
jpgraph/etc., and also because www.open-mpi.org is used for other  
server purposes).  So moving the data to the Google Apps datastore  
*could* give us a better underlying platform.


A further thought is that perhaps we should roll up all the pending  
ideas we have for MTT (and there are a lot of them ;-) -- to include  
the one described above) and apply for a Google Summer of Code  
student.


   http://code.google.com/soc/

GSoC applications can be submitted March 9-13 2009.  This sounds  
like it could be a winner...


Want to start a wiki page with a list of GSoC ideas?




On Feb 24, 2009, at 5:06 PM, Jeff Squyres (jsquyres) wrote:

On Feb 24, 2009, at 4:49 PM, Josh Hursey wrote:

>> Should we allow direct postgres connections (across the internet)
>> to specific OMPI organizations who want/need it?
>
> It is possible that we could allow read-only access to specific
> organizations. I would be extremely careful about granting write
> access.

Agreed; I think we should only allow read-only to specific IP  
addresses.


It sounds like this *might* solve some of the issues (assuming they
want to take the time to figure out the schema).  Should we explore
this possibility?

(the SQL schemas are in the MTT SVN repo, if you didn't know that
already)

--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] visual reports for mtt

[Jeff Squyres]

Just to bring the info to the list...

There have been some off list e-mails and some in-person discussions  
about a fascinating idea that looks promising here.


How about moving the MTT data store to the Google Apps Engine/ 
datastore?  See here:


http://code.google.com/appengine/docs/whatisgoogleappengine.html

Josh looked at this about a year ago and thought that there would be  
some interesting possibilities here, but never had the cycles to  
follow up on it.  Specifically: if we move all the MTT data from a  
postgres DB to the Google Apps datastore, we might have a highly  
scalable mechanism for queries, and therefore be able to do much more  
interesting kinds of queries (right now, we're fairly limited in our  
queries because of memory and CPU restrictions via Apache/PHP/jpgraph/ 
etc., and also because www.open-mpi.org is used for other server  
purposes).  So moving the data to the Google Apps datastore *could*  
give us a better underlying platform.


A further thought is that perhaps we should roll up all the pending  
ideas we have for MTT (and there are a lot of them ;-) -- to include  
the one described above) and apply for a Google Summer of Code student.


http://code.google.com/soc/

GSoC applications can be submitted March 9-13 2009.  This sounds like  
it could be a winner...


Want to start a wiki page with a list of GSoC ideas?



On Feb 24, 2009, at 5:06 PM, Jeff Squyres (jsquyres) wrote:


On Feb 24, 2009, at 4:49 PM, Josh Hursey wrote:

>> Should we allow direct postgres connections (across the internet)
>> to specific OMPI organizations who want/need it?
>
> It is possible that we could allow read-only access to specific
> organizations. I would be extremely careful about granting write
> access.

Agreed; I think we should only allow read-only to specific IP  
addresses.


It sounds like this *might* solve some of the issues (assuming they
want to take the time to figure out the schema).  Should we explore
this possibility?

(the SQL schemas are in the MTT SVN repo, if you didn't know that
already)

--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] visual reports for mtt

[Jeff Squyres]

On Feb 24, 2009, at 4:49 PM, Josh Hursey wrote:

Should we allow direct postgres connections (across the internet)  
to specific OMPI organizations who want/need it?


It is possible that we could allow read-only access to specific  
organizations. I would be extremely careful about granting write  
access.


Agreed; I think we should only allow read-only to specific IP addresses.

It sounds like this *might* solve some of the issues (assuming they  
want to take the time to figure out the schema).  Should we explore  
this possibility?


(the SQL schemas are in the MTT SVN repo, if you didn't know that  
already)


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT bugs] [MTT] #258: Compare non-contiguous date ranges

[Jeff Squyres]

Hey, how about Javascript calendars for selecting date ranges?

:p


On Nov 11, 2008, at 1:15 PM, Ethan Mallove wrote:


Folks,

I have a fix for this ticket. The syntax to get
non-contiguous date ranges is to use space-delimited quoted
tokens (see http://tinyurl.com/6e2pqa), e.g.,

 "2007-10-29 - 2007-11-30" "2008-11-07 - 2008-11-14" "by month"

Trying to view year-old results alongside week-old results in the
current Reporter can result in 10+ minutes of database thrashing. With
this fix, it's around 30 seconds. If I don't get any bug reports about
the above ~emallove Reporter within 48 hours, I'll push it out to the
main www.open-mpi.org site.

Thanks,
Ethan


On Wed, Aug/08/2007 07:22:14PM, MTT wrote:

#258: Compare non-contiguous date ranges
- 
+--

Reporter:  jjhursey |   Owner:
Type:  defect   |  Status:  new
Priority:  major|   Milestone:  Future
Component:  Server side  | Version:  trunk
Keywords:   |
- 
+--
It is possible to get results from two types of runs (say threaded  
and
non-threaded) by regular expression arguments to some existing  
reporter

fields (i,e., configure arguments).

However it is currently not possible to compare two distinct date  
ranges.

We should extend the reporter to effectively display such results.

We could extend the date specification to allow for a comma  
separated list

of ranges:
{{{
To compare results from July 20, 21, and 22:
  2007-07-20 - 2007-07-21,2007-07-21 - 2007-07-22,2007-07-22 -  
2007-07-23

}}}

Then the reporter could make distinct summary tables for each  
range. Or

something like that.

--
Ticket URL: <https://svn.open-mpi.org/trac/mtt/ticket/258>
MTT <http://www.open-mpi.org/>
Issue tracking for the MPI Testing Tool.




--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mpi_details section with different scenarios for command line params

[Jeff Squyres]

On Nov 4, 2008, at 7:35 AM, Mike Dubman wrote:

yep. it works. I thought that "exec" for mpirun will be executed  
once with all @mca@ params passed to it.


I basically have a top-level "if" that lets me figure out whether I'm  
running on IB or iWARP systems, and that chooses between two master  
sets of MCA parameters.


Let me know if you have any questions about that INI file.

FYI, Ethan and I reflect quite different ways of running MTT:

- he likes scripting and putting together pieces of INI on the fly  
(command line params, etc.)

- I like having one big INI file and selectively running parts of it

Both are completely compatible with MTT, and both are good models.  We  
tried to make MTT flexible enough to be able to handle such different  
models as these two (as another approach, Indiana U. builds INI files  
on the fly).


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mpi_details section with different scenarios for command line params

[Jeff Squyres]

What exactly do you want to do?

For example, Cisco's MTT files simply list a huge number of different  
mpirun command lines in the MPI Details section (25, in one case,  
IIRC).  So I run lots of different cases for each MPI test (e.g., with  
leave pinned, without leave pinned, ...etc.).



On Nov 3, 2008, at 10:45 AM, Ethan Mallove wrote:


On Mon, Nov/03/2008 09:34:07AM, Mike Dubman wrote:

  Hello Guys,

  Please suggest the proper way to handle the following:

  Is there any way to run "test run" section with a list
  of "mpi_details" sections?


Mike,

There is currently no way to iterate over multiple
mpi_details sections, but there might be an acceptable
workaround. You can create a simple wrapper script to
iterate over variations of your MPI details section using
command line INI file overrides (see
https://svn.open-mpi.org/trac/mtt/wiki/INIOverrides). E.g.,
say you have the following MPI details section:

 [MPI details: Open MPI]
 foo = some default value
 bar = some default value
 exec = mpirun @foo@ @bar@ ...

Using command-line INI overrides, you can iterate over a
series of values for "foo" and/or "bar":

 $ client/mtt --scratch /some/dir ...
 $ client/mtt --scratch /some/dir --test-run foo=abc ...
 $ client/mtt --scratch /some/dir --test-run foo=def ...
 $ client/mtt --scratch /some/dir --test-run bar=uvw ...
 $ client/mtt --scratch /some/dir --test-run bar=xyz ...
 ...

Note in the above example, we use the same scratch directory
for all the runs, and we run only the test run phase (via
the --test-run option) since we do not need to reinstall or
rebuild anything as we iterate over different command lines.

Could the above be of use for what you're trying to do?

-Ethan




  Or how to execute specific "Test run" section against
  specific "mpi_details" section, where "mpi_details" can
  have many different scenarios of command line
  parameters (i.e. single mpi_details should be executed
  a number of times equal to the number of available
  scenarios for this mpi_details)? Is that possible? (it
  is similar to the @np param treatment available inside
  mpi_details section)

  Thanks

  Mike.



___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

I sent the whatami patch upstream, and Brian Finley (the whatami  
author) encouraged us to actually use his recently-included Centos-5  
support instead of our patch.  This is because his support is generic  
enough that it should work for any lsb_release-capable machine (to  
include Centos 5).


I pulled that down into the MTT trunk; Mike, could you verify that it  
works for you?



On Oct 28, 2008, at 8:30 AM, Jeff Squyres (jsquyres) wrote:


Done!

On Oct 28, 2008, at 2:06 AM, Mike Dubman wrote:

>
> Hey Jeff,
>
> I have no svn permissions to commit. Can you please provide me with
> one? (login: miked)
> Thanks
>
> On Mon, Oct 27, 2008 at 4:38 PM, Jeff Squyres <jsquy...@cisco.com>
> wrote:
> Aside from the 2 space tabs, looks great.  ;-)
>
> Go ahead and commit; I'll send the whatami patch upstream (whatami
> is maintained by Brian Finley at Argonne National Labs).
>
>
>
> On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:
>
>
> Hello guys,
>
> Please consider applying attached mtt patch to allow following
> features:
>
>• Support for centos5
>• Send single, digested email report for all completed tests
> (similar to text file summary file)
>• Provide basic statistics in the digested email about
> completed tests (similar to junit): duration, mpi version, overall
> status.
>
> Thanks
>
> Mike
> 
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
>
> --
> Jeff Squyres
> Cisco Systems
>
>
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
>
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

Done!

On Oct 28, 2008, at 2:06 AM, Mike Dubman wrote:



Hey Jeff,

I have no svn permissions to commit. Can you please provide me with  
one? (login: miked)

Thanks

On Mon, Oct 27, 2008 at 4:38 PM, Jeff Squyres <jsquy...@cisco.com>  
wrote:

Aside from the 2 space tabs, looks great.  ;-)

Go ahead and commit; I'll send the whatami patch upstream (whatami  
is maintained by Brian Finley at Argonne National Labs).




On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:


Hello guys,

Please consider applying attached mtt patch to allow following  
features:


   • Support for centos5
   • Send single, digested email report for all completed tests  
(similar to text file summary file)
   • Provide basic statistics in the digested email about  
completed tests (similar to junit): duration, mpi version, overall  
status.


Thanks

Mike


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


--
Jeff Squyres
Cisco Systems



___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt patch: summary digest

[Jeff Squyres]

Aside from the 2 space tabs, looks great.  ;-)

Go ahead and commit; I'll send the whatami patch upstream (whatami is  
maintained by Brian Finley at Argonne National Labs).



On Oct 26, 2008, at 10:14 AM, Mike Dubman wrote:



Hello guys,

Please consider applying attached mtt patch to allow following  
features:


• Support for centos5
	• Send single, digested email report for all completed tests  
(similar to text file summary file)
	• Provide basic statistics in the digested email about completed  
tests (similar to junit): duration, mpi version, overall status.


Thanks

Mike

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] bogus timestamps in database

[Jeff Squyres]

I can do 3-3:30 today.


On Jul 22, 2008, at 10:19 AM, Josh Hursey wrote:

I think we were scheduled for talking about MTT today at 1 pm.  
Something has come up and I can't do it during that time. I can meet  
from 3 - 3:30 today or tomorrow 10-11:30, 2-3:30. I can also do  
anytime on Thursday.


Sorry for the move. :(

-- Josh

On Jul 21, 2008, at 1:35 PM, Jeff Squyres wrote:


Ok; I'll 3-way call you guys -- no need for a phone bridge.

On Jul 21, 2008, at 1:32 PM, Ethan Mallove wrote:


3p works for me.

-Ethan

On Mon, Jul/21/2008 09:52:21AM, Jeff Squyres wrote:

Sorry for the late reply -- how about 3pm today (Monday)?


On Jul 18, 2008, at 3:55 PM, Ethan Mallove wrote:


I'm basically available except for 2-4p on Friday.

-Ethan

On Fri, Jul/18/2008 10:02:10AM, Josh Hursey wrote:
Yeah it might be good to touch base and see if there are any  
burning

fires
that we need to put out.

For me it would have to be either next week or most likely the  
end of
August. :( The good news is that I don't have too much on my  
calendar for

next week. I'm available (prefer earlier in the week):
M - Anytime
T - 12 - 2, 2-4
W - 10 - 11:30, 2 - 4
R - anytime
F - not available.

-- Josh

On Jul 18, 2008, at 9:29 AM, Jeff Squyres wrote:

Mebbe we should have a teleconf sometime in the not-distant  
future and

see
if we want to prioritize some of the pending MTT work...?


On Jul 17, 2008, at 6:39 PM, Ethan Mallove wrote:


On Thu, Jul/17/2008 04:35:38PM, Jeff Squyres wrote:

Here's a fun report (as of 17 July 2008):

http://www.open-mpi.org/mtt/index.php?do_redir=775

Note that two of the rows are in the future.  :-)  (Absoft  
has since

fixed
the problem; ntp accidentally got turned off)

Ethan and I talked about this a bit, and then Josh and I  
talked about

it.
Posting here to summarize everything:

- It seems like a good solution for the moment is for  
submit.php to

examine
all the timestamps in a given submit and compare them to  
now().  Find

the
timestamp that latest in time, and compute  
x=latest_timestamp - now().

If
x>0, then subtract x from *all* timestamps in the submitted  
data.

Then
print a Big Hairy Warning on the client that their time is not
coordinated
with the server.

- Josh thinks that we should have a larger conversation  
about how to

have
some values be regulated (e.g., MPI name, test suite name,  
etc.).  I

agree
-- classic case: some people call it "intel", others call  
it"intelsuite".They show up differently in the DB.  My $0.02  
is that we

should
allow
people to call it whatever they want in the .ini file, but  
then

somehow
ensure to submit the names all consistently (e.g., ini file  
has a map

of
"this ini section is reported as 'intel'"), and if the name is
invalid,
reject the data from the DB (or maybe put it in  
"quarrantine" so that

it
can be cleaned up and put in the main DB)?



It's a neccessity that the INI section naming be flexible.
E.g., I have intel-32 and intel-64 sections to test 32-bit
and 64-bit. Note, there's no way around this because the
Perl INI parser we're using does not allow duplicate section
names, and also, MTT constructs a scratch tree on the
assumption that each section has a unique name.

For "database text string regulation", there's also:

http://svn.open-mpi.org/trac/mtt/ticket/7 (users should be
able to delete or modify results from database)

-Ethan


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] bogus timestamps in database

[Jeff Squyres]

Mebbe we should have a teleconf sometime in the not-distant future and  
see if we want to prioritize some of the pending MTT work...?



On Jul 17, 2008, at 6:39 PM, Ethan Mallove wrote:


On Thu, Jul/17/2008 04:35:38PM, Jeff Squyres wrote:

Here's a fun report (as of 17 July 2008):

   http://www.open-mpi.org/mtt/index.php?do_redir=775

Note that two of the rows are in the future.  :-)  (Absoft has  
since fixed

the problem; ntp accidentally got turned off)

Ethan and I talked about this a bit, and then Josh and I talked  
about it.

Posting here to summarize everything:

- It seems like a good solution for the moment is for submit.php to  
examine
all the timestamps in a given submit and compare them to now().   
Find the
timestamp that latest in time, and compute x=latest_timestamp -  
now().  If
x>0, then subtract x from *all* timestamps in the submitted data.   
Then
print a Big Hairy Warning on the client that their time is not  
coordinated

with the server.

- Josh thinks that we should have a larger conversation about how  
to have
some values be regulated (e.g., MPI name, test suite name, etc.).   
I agree
-- classic case: some people call it "intel", others call it "intel  
suite". 
They show up differently in the DB.  My $0.02 is that we should allow
people to call it whatever they want in the .ini file, but then  
somehow
ensure to submit the names all consistently (e.g., ini file has a  
map of
"this ini section is reported as 'intel'"), and if the name is  
invalid,
reject the data from the DB (or maybe put it in "quarrantine" so  
that it

can be cleaned up and put in the main DB)?



It's a neccessity that the INI section naming be flexible.
E.g., I have intel-32 and intel-64 sections to test 32-bit
and 64-bit. Note, there's no way around this because the
Perl INI parser we're using does not allow duplicate section
names, and also, MTT constructs a scratch tree on the
assumption that each section has a unique name.

For "database text string regulation", there's also:

 http://svn.open-mpi.org/trac/mtt/ticket/7 (users should be
 able to delete or modify results from database)

-Ethan


--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

[MTT devel] bogus timestamps in database

[Jeff Squyres]

Here's a fun report (as of 17 July 2008):

http://www.open-mpi.org/mtt/index.php?do_redir=775

Note that two of the rows are in the future.  :-)  (Absoft has since  
fixed the problem; ntp accidentally got turned off)


Ethan and I talked about this a bit, and then Josh and I talked about  
it.  Posting here to summarize everything:


- It seems like a good solution for the moment is for submit.php to  
examine all the timestamps in a given submit and compare them to  
now().  Find the timestamp that latest in time, and compute  
x=latest_timestamp - now().  If x>0, then subtract x from *all*  
timestamps in the submitted data.  Then print a Big Hairy Warning on  
the client that their time is not coordinated with the server.


- Josh thinks that we should have a larger conversation about how to  
have some values be regulated (e.g., MPI name, test suite name,  
etc.).  I agree -- classic case: some people call it "intel", others  
call it "intel suite".  They show up differently in the DB.  My $0.02  
is that we should allow people to call it whatever they want in  
the .ini file, but then somehow ensure to submit the names all  
consistently (e.g., ini file has a map of "this ini section is  
reported as 'intel'"), and if the name is invalid, reject the data  
from the DB (or maybe put it in "quarrantine" so that it can be  
cleaned up and put in the main DB)?


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT bugs] [MTT] #355: tooltips for reporter

[Jeff Squyres]

Ethan and I played with this a bit more and it might not be workable.   
See #355 for more details.


On Apr 21, 2008, at 2:06 PM, Ethan Mallove wrote:


Do these work for you?

 http://tinyurl.com/49m2n4

They work for me in IE (Windows) and Mozilla (Solaris), but
not in Firefox and Opera. The joys of JavaScript :-)

-Ethan


On Mon, Apr/21/2008 01:32:49PM, MTT wrote:

#355: tooltips for reporter
- 
+--

Reporter:  jsquyres |   Owner:
Type:  enhancement  |  Status:  new
Priority:  major|   Milestone:  v3.1
Component:  Server side  | Version:  trunk
Keywords:   |
- 
+--
Currently, when browsing the reporter, if you hover over a link in  
the

reporter, the browser (or mail client) may show the entire URL of the
hyperlink as a tooltip (see attached screenshot).  Is there a way  
to show
something nicer / smaller?  In the attached screenshot, either no  
tooltip
or a tooltip showing "Drill down to all 1.2.7a results" would  
be nice.


--
Ticket URL: <https://svn.open-mpi.org/trac/mtt/ticket/355>
MTT <http://www.open-mpi.org/>
Issue tracking for the MPI Testing Tool.


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1176

[Jeff Squyres]

 if ($test_get_name eq "all" ||
+$test_get_key eq $test_get_name) {
my $test_get = $MTT::Test::sources- 
>{$test_get_key};


# For each MPI source

Modified: trunk/lib/MTT/Test/Run.pm
= 
= 
= 
= 
= 
= 
= 
= 
= 
=

--- trunk/lib/MTT/Test/Run.pm   (original)
+++ trunk/lib/MTT/Test/Run.pm	2008-04-04 15:31:07 EDT (Fri, 04 Apr  
2008)

@@ -105,7 +105,8 @@

# This is only warning about the INI file; we'll see
# if we find meta data for the test build later
-if (!$ini_full->SectionExists("test build:  
$test_build_name")) {

+if ($test_build_name ne "all" &&
+!$ini_full->SectionExists("test build:  
$test_build_name")) {
Warning("Test Build section \"$test_build_name 
\" does not seem to exist in the INI file\n");

}

@@ -130,7 +131,8 @@
last
if  
(MTT::Util::find_terminate_file());


-if ($test_build_key eq  
$test_build_name) {

+if ($test_build_name eq "all" ||
+$test_build_key eq  
$test_build_name) {
my $test_build =  
$MTT::Test::builds->{$mpi_get_key}->{$mpi_version_key}- 
>{$mpi_install_key}->{$test_build_key};
Debug("Found a match!  
$test_build_key [$simple_section\n");
        if (!$test_build- 
>{test_result}) {

___
mtt-svn mailing list
mtt-...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] Launch scaling data in MTT

[Jeff Squyres]

MTT probably could gather this data -- some of it was wall clock  
execution time; other was multiple parts of data extracted from stdout.


Ralph -- is this interesting / useful to you?

On Apr 4, 2008, at 4:56 PM, Ethan Mallove wrote:

I was looking at the graphs posted at
https://svn.open-mpi.org/trac/ompi/wiki/ORTEScalabilityTesting,
and noted that MTT could gather this data if it could just
graph on the "duration" column. Would this be useful? E.g.,

 http://www.open-mpi.org/mtt/index.php?do_redir=582

Note: the duration timings currently round to the second,
but the "duration" column uses an interval type that has
microsecond precision, if we needed it.

-Ethan
___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1164

[Jeff Squyres]

On Mar 20, 2008, at 12:19 PM, Ethan Mallove wrote:

I sense this "do_not_run" stuff could be useful to me, but
I'm not sure. Can you give a simple use case for
"do_not_run"? Could "do_not_run" be achieved by just
commenting out the INI lines that pertain to a certain
group? E.g., do these "#" comments ...

 # simple_really_slow:tests = src/MPI_Isend_flood_c src/ 
MPI_Send_flood_c


No, because if an upper-level _executables() found  
MPI_*end_flood_c, then they'll run in that section (vs. the  
simple_really_slow subsection).  The intent was to have a mechanism to  
specify: here's some tests, exclude them from all other sections, but  
*don't* run them.




 # simple_really_slow:pass = (_wifexited(),  
(_wexitstatus(), 0))

 # simple_really_slow:exclusive = 1
 # simple_really_slow:timeout = (90, (3, _np()))

... have the same effect as this?

 simple_really_slow:do_not_run = 1


Instead, you'd have:

simple_really_slow:tests = src/MPI_Isend_flood_c src/MPI_Send_flood_c
simple_really_slow:do_not_run = 1

--
Jeff Squyres
Cisco Systems

Re: [MTT devel] [MTT svn] svn:mtt-svn r1163

[Jeff Squyres]

 "trivial_tests_cflags");
my $fflags = Value($ini, $config->{full_section_name},
   "trivial_tests_fflags");
+my $languages = Value($ini, $config->{full_section_name},
+   "trivial_tests_languages");
+
+# Default to running *all* flavors of trivial tests
+if (!$languages) {
+$languages = "c,c++,f77,f90";
+}
+
+my $languages_hash;
+my @arr = split(/,|\s+/, $languages);
+foreach my $lang (@arr) {
+$lang = lc($lang);
+$languages_hash->{"$lang"} = 1;
+}

# Try compiling and linking a simple C application

-if ($mpi_install->{c_bindings}) {
+if ($mpi_install->{c_bindings} and $languages_hash->{"c"}) {
Debug("Test compile/link sample C MPI application\n");
$x = _do_compile("mpicc $cflags", "hello.c", "c_hello");
return $x
@@ -73,7 +87,7 @@
# If we have the C++ MPI bindings, try and compile and link a
# simple C++ application

-if ($mpi_install->{cxx_bindings}) {
+if ($mpi_install->{cxx_bindings} and $languages_hash->{"c++"}) {
Debug("Test compile/link sample C++ MPI application\n");
$x = _do_compile("mpic++ $cflags", "hello.cc", "cxx_hello");
return $x
@@ -88,7 +102,7 @@
# If we have the F77 MPI bindings, try compiling and linking a
# simple F77 application

-if ($mpi_install->{f77_bindings}) {
+if ($mpi_install->{f77_bindings} and $languages_hash->{"f77"}) {
Debug("Test compile/link sample F77 MPI application\n");
$x = _do_compile("mpif77 $fflags", "hello.f", "f77_hello");
return $x
@@ -103,7 +117,7 @@
# If we have the F90 MPI bindings, try compiling and linking a
# simple F90 application

-if ($mpi_install->{f90_bindings}) {
+if ($mpi_install->{f90_bindings} and $languages_hash->{"f90"}) {
Debug("Test compile/link sample F90 MPI application\n");
$x = _do_compile("mpif90 $fflags", "hello.f90", "f90_hello");
return $x
___
mtt-svn mailing list
mtt-...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-svn



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] MTT Visualization

[Jeff Squyres]

On Jan 10, 2008, at 10:29 AM, Josh Hursey wrote:


I met with Joseph Cottam (Grad student in my lab at IU) yesterday
about MTT visualization. He is working on some new visualization
techniques and wants to apply them to the MTT dataset.


Awesome.


Since we are ramping up to a v1.3 release we want to visualization to
support this effort. So we want to make sure that the visualization
will meet the development community's needs. We should probably ask
the devel-core list, but I thought I would start some of the
discussion here to make sure I am asking the right questions of the
group.


Sounds reasonable.

After a first go-round here, we might want to have a conversation with  
the OMPI RM's to get their input - that would still be a small group  
to get targeted feedback on these questions.



To start I have some basic questions:
 - How does Open MPI determine that it is stable enough to release?


I personally have a Magic 8 Ball on my desk that I consult frequently  
for questions like this.  ;-)


It's a mix of many different metrics, actually:

- stuff unrelated to MTT results:
   - how many trac tickets are open against that release and do we care
   - how urgent are the bug fixes that are included
   - external requirements (e.g., get an OMPI release out to meet the  
OFED release schedule)

   - ...and probably others
- related to MTT results
   - "good" coverage on platforms (where "platform" = host arch, OS,  
OS version, compiler, compiler version, MCA params, interconnect, and  
scheduler -- note that some of these are orthogonal from each other...)
   - the only failures and timeouts we have are a) repeatable, b)  
consistent across multiple organizations (if relevant), and deemed to  
be acceptable



 - What dimensions of testing are most/least important (i.e.,
platforms, compilers, feature sets, scale, ...)?


This is a hard question.  :-\  I listed several dimensions above:

- host architecture
- OS
- OS version
- compiler
- compiler version
- MCA parameters used
- interconnect
- scheduler

Here's some more:

- number of processes tested
- layout of processes (by node, by proc, ...etc.)

I don't quite know how to order those in terms of priority.  :-\


 - What other questions would be useful to answer with regard to
testing (thinking completely outside of the box)?
   * Example: Are we testing a specific platform/configuration set
too much/too little?


This is a great question.

I would love to be able to configure this question -- e.g., are we  
testing some MCA params too much/too little.


The performance stuff can always be visualized better, especially over  
time. One idea is expressed in https://svn.open-mpi.org/trac/mtt/ticket/330 
. 

I also very much like the ideas in https://svn.open-mpi.org/trac/mtt/ticket/236 
 and https://svn.open-mpi.org/trac/mtt/ticket/302 (302 is not  
expressed as a visualization issue, but it could be -- you can imagine  
a tree-based display showing the relationships between phase results,  
perhaps even incorporated with a timeline -- that would be awesome).


Here's a whacky idea -- can our MTT data be combined with SCM data  
(SVN, in this case) to answer questions like:


- what parts of the code are the most troublesome?  i.e., when this  
part of the code changes, these tests tend to break


- what tests seem to be related to what parts of the OMPI code base?

- who / what SVN commit(s) seemed to cause specific tests to break?

(this seems like a longer-term set of questions, but I thought I'd  
bring it up...)



 - Other questions you think we should pose to the group?

We are currently feeling out the domain of possibilities, but hope to
start doing some sketching some ideas in another week or so. This work
should proceed fairly quickly since we are targeting a paper about
this for the ACM Symposium on Software Visualization (http://www.softvis.org/
) which is due in early April. How is that for expecting success :)


Awesome.

--
Jeff Squyres
Cisco Systems

[MTT devel] server side Mellanox patch

[Jeff Squyres]

Pasha sent a 2nd file with their server-side changes.  It looks like  
there is only one substantive change in report.inc:


-do_pg_query("set sort_mem = '256MB';");
+do_pg_query("set sort_mem = 256;");

Pasha -- what was this change for?  Does a knob for this change need  
to be added to config.inc?


--
Jeff Squyres
Cisco Systems

Re: [MTT devel] Oleg's mtt client changes.

[Jeff Squyres]

Pasha sent me this list of changes that Mellanox made to the client  
side of MTT.  Some of them have already been accepted back into the  
trunk, so I snipped those from this mail.  There's two main fixes  
left: copytree and new functionality in the Mail reporter.


More comments below.


On Jan 10, 2008, at 10:21 AM, Pavel Shamis wrote:


Jeff, please see the attached file.
Most of the fixes already fixed by mtt.


Please see below my comments (grep )

Index: lib/MTT/MPI/Install/Copytree.pm
===
--- lib/MTT/MPI/Install/Copytree.pm (revision 1123)
+++ lib/MTT/MPI/Install/Copytree.pm (working copy)
@@ -48,7 +48,7 @@ sub Install {
 $ret->{exit_status} = 1;
 $ret->{result_message} = "MPI INSTALL COPYTREE PLUGIN IS OUT OF  
DATE.  CONTACT AUTHORS";

 Verbose("*** $ret->{result_message}\n");
-return $ret;
+#return $ret;


Hah -- I guess the above message says it all, eh?  :-)




 Debug(">> copytree copying to $config->{installdir}\n");
@@ -69,9 +69,18 @@ sub Install {
 }

 # Copy the tree
-MTT::DoCommand::Pushdir($config->{installdir});
+#MTT::DoCommand::Pushdir($config->{installdir});
+#$x = MTT::Files::copy_tree("$config->{abs_srcdir}", 1);
+#MTT::DoCommand::Popdir();
+#return undef
+#if (!$x);
+
+#OLEG#
+# Copy the tree
+my $start_dir = cwd();
+MTT::DoCommand::Chdir($config->{installdir});
 $x = MTT::Files::copy_tree("$config->{abs_srcdir}", 1);
-MTT::DoCommand::Popdir();
+MTT::DoCommand::Chdir($start_dir);
 return undef
 if (!$x);
 i guess it is copytree fixes


Looks mostly good; I'll commit (and remove the other "this is broken!"  
stuff).



Index: lib/MTT/Mail.pm
===
--- lib/MTT/Mail.pm (revision 1123)
+++ lib/MTT/Mail.pm (working copy)
@@ -54,7 +54,9 @@ sub Init {
  
#--


 sub Send {
-my ($subject, $to, $body) = @_;
+#my ($subject, $to, $body) = @_;
+#OLEG#
+my ($subject, $to, $from, $body) = @_;

 Init()
 if (! $initialized);
@@ -66,11 +68,12 @@ sub Send {

 # Invoke the mail agent to send the mail

-open MAIL, "|$mail_agent -s \"$subject\" \"$to\"" ||
+#open MAIL, "|$mail_agent -s \"$subject\" \"$to\"" ||
+#OLEG#
+open MAIL, "|$mail_agent -r \"$from\" -s \"$subject\" \"$to\"" ||
 die "Could not open pipe to output e-mail\n";
 print MAIL "Subject: $subject\n";
 print MAIL "$body\n";
-close MAIL;

 # Restore the old environment

 Here he is changing default user from to some other value.


This looks reasonable, but I think we only want to add the -r if $from  
is defined.



 # If we have a hostlist, return its max procs count
Index: lib/MTT/Reporter/Email.pm
===
--- lib/MTT/Reporter/Email.pm (revision 1123)
+++ lib/MTT/Reporter/Email.pm (working copy)
@@ -81,6 +81,8 @@ sub Submit {

 # Evaluate the email subject header
 my $subject = Value($ini, $section, "email_subject");
+#OLEG#
+my $from = Value($ini, $section, "email_from");

 my $s;
 my $body;
@@ -116,7 +118,9 @@ sub Submit {
 }

 # Now send it
-MTT::Mail::Send($s, $to, $body);
+#MTT::Mail::Send($s, $to, $body);
+#OLEG#
+MTT::Mail::Send($s, $to, $from, $body);
 Verbose(">> Reported to e-mail: $to\n");
 }

 again mail fixes



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] mtt reporter popups broken

[Jeff Squyres]

Perfect -- I can confirm that it all works again.

Thanks!


On Jan 10, 2008, at 12:53 PM, Josh Hursey wrote:


Fixed. This was a latent bug that my commit highlighted, but I didn't
check for (Sorry). If mtt_body_html_suffix was non-null then the
popups would not work. I just fixed this in r1133.

-- Josh

On Jan 10, 2008, at 12:26 PM, Jeff Squyres wrote:


I notice that I can no longer get the MTT preferences/advanced popups
anymore in Safari.  They were working earlier this morning -- did
Josh's commit break something, perchance?

--
Jeff Squyres
Cisco Systems

___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel


___
mtt-devel mailing list
mtt-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel



--
Jeff Squyres
Cisco Systems