Re: [MTT devel] Cisco submits to Josh DB
jms-new-parser is part of v3.0 too, right? I ran into issues with it: It's doing some funky things w/ my command: http://osl.iu.edu/~jjhursey/research/mtt/server/php/reporter.php?do_redir=7 It repeats the command string 5 times in some spots. And should "network" and "runtime params" be blank for: mpirun --mca btl self,tcp --host burl-ct-v440-4,burl-ct-v440-4,burl-ct-v440-4,burl-ct-v440-4,burl-ct-v440-5,burl-ct-v440-5,burl-ct-v440-5,burl-ct-v440-5 -np 2 --prefix /opt/SUNWhpc/HPC7.0/ ./c_hello ? -Ethan On Thu, Aug/23/2007 04:06:39PM, Josh Hursey wrote: > Yeah I just ran IU's nightly set against my DB and it went fine > submitting 6024 test_run results. > I say we are good to go forward. > > Here are some performance numbers: > Summary Report (24 hours all orgs) > [ 107,070 test_runs] > v2 - 87 sec > v3 - 6 sec > Summary Report (24 hours Only 'iu') > [ 29,625 test_runs] > v2 - 37 sec > v3 - 4 sec > > Summary Report (Past 3 days all orgs) > [ 294,395 test_runs] > v2 - 138 sec > v3 - 9 sec > > Summary Report (Past 3 days Only 'iu') > [ 86,026 test_runs] > v2 - 49 sec > v3 - 11 sec > > Summary Report (Past 2 weeks all orgs) > [ 1,460,824 test_runs] > v2 - 863 sec > v3 - 34 sec > Summary Report (Past 2 weeks Only 'iu') > [ 346,804 test_runs] > v2 - 878 sec (not seeded) > v2 - 2 sec (pre-seeded) > v3 - 12 sec > > > Summary Report (Past 1 month all orgs) > [2,981,678 test_runs] > v2 - 1395 sec > v3 - 158 sec > Summary Report (Past 1 month Only 'iu') > [ test_runs] > v2 - 1069 sec (not seeded) > v2 -2 sec (pre-seeded) > v3 - 39 sec > > Summary Report (2007-06-18 - 2007-06-19 all orgs) > [ 43,816 test_runs] > v2 - 484 sec > v3 - 5 sec > Summary Report (2007-06-18 - 2007-06-19 only 'iu') > [ 30,059 test_runs] > v2 - 479 sec (not seeded) > v2 - 2 sec (pre-seeded) > v3 - 2 sec > > > Some stats I collected that we need not distribute, but thought were > interesting: > Some intersting stats to date (2007-08-20) > # test_run results : 17,522,720 > # test_build results : 42,655 > # mpi_install results: 45,096 > # of performance results : 58,365 > > Most popular test suites: >intel: 65.5 % >ibm : 28.0 % >onesided : 4.3 % >trivial : 1.5 % >imb : 0.4 % > > Most popular compilers: >gnu : 49.8 % >intel : 30.8 % >sun : 6.9 % >pathscale : 6.8 % >pgi : 5.7 % > > OS testing percentages: >Linux: 93.2 % >SunoS: 6.8 % > > Org. Contribution percentages: >Cisco: 66.4 % >IU : 23.2 % >Sun : 6.8 % >IBM : 2.3 % >HLRS : 1.3 % >Voltaire : 0.0 % >UTK : 0.0 % > > > On Aug 23, 2007, at 12:35 PM, Jeff Squyres wrote: > > > I think we got good submits to the josh db last night -- I think my > > prior problems were pilot error (accidentally sharing scratch > > directories between my josh-db testing MTT runs and my production MTT > > runs). > > > > http://osl.iu.edu/~jjhursey/research/mtt/server/php/reporter.php? > > do_redir=6 > > > > So I think Cisco is happy with the new DB and good for the Monday > > rollout plan. > > > > -- > > Jeff Squyres > > Cisco Systems > > > > ___ > > mtt-devel mailing list > > mtt-de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel > > ___ > mtt-devel mailing list > mtt-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
Re: [MTT devel] Cisco submits to Josh DB
On Aug 24, 2007, at 9:46 AM, Ethan Mallove wrote: jms-new-parser is part of v3.0 too, right? I hope so. :-) I ran into issues with it: It's doing some funky things w/ my command: http://osl.iu.edu/~jjhursey/research/mtt/server/php/reporter.php? do_redir=7 I just filed a ticket about this: https://svn.open-mpi.org/trac/ompi/ ticket/1141 -- can you upload the ini file that you used to generate this test run? It repeats the command string 5 times in some spots. And should "network" and "runtime params" be blank for: mpirun --mca btl self,tcp --host burl-ct-v440-4,burl-ct- v440-4,burl-ct-v440-4,burl-ct-v440-4,burl-ct-v440-5,burl-ct- v440-5,burl-ct-v440-5,burl-ct-v440-5 -np 2 --prefix /opt/SUNWhpc/ HPC7.0/ ./c_hello You need to add these fields into your mpi details section: parameters = &MPI::OMPI::find_mpirun_params(&test_command_line(), \ &test_executable()) network = &MPI::OMPI::find_network(&test_command_line(), &test_executable()) Obviously, they're OMPI-specific funclets. -- Jeff Squyres Cisco Systems
[MTT devel] MTT Database and Reporter Upgrade **Action Required**
Guys take a look at this email, and let me know if there is anything missing before we sent it out. The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. At the bottom of this email are some rough numbers showing the difference between the current and the new versions. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am EDT and we hope to have it back by Noon EDT. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new
[MTT devel] Testing of perf bakeoff template INI
Josh asked me to send some details on the "collectives performance bakeoff" INI template file that its in the jms branch samples/ompi- core-perf-testing.ini. Here's the scoop: * As usual, this is a template. There are values that you will need to fill in (e.g., the MTT database username/password, the MPICH- MX username/password, etc.), and values that you will need to tweak for your site. * The easy part: the test get, build, and run sections are for the following tests: netpipe, OSU, IMB, and SKaMPI. It's actually a far smaller test set than we run for regression / correctness testing. The SKaMPI tests that are there right now are preliminary; Jelena will be making up a new set next week sometime. But testing with what is there now is still most useful (to verify MTT's functionality). * I added support for many more MPI's to MTT; this is what has consumed the majority of my time this week. Here's the MPI's that we currently support: - Open MPI (of course) - MPICH1, MPICH2 (still waiting on word on a legal issue about a patch for MPICH1 to run properly under SLURM) - MVAPICH1, MVAPICH2 - Intel MPI - HP MPI (should be done this afternoon) * Other MPIs that will likely not be difficult to add (I do not have access to do the work myself): - Scali MPI - Cray MPI - MPICH-MX - CT6 / CT7 * The MPI get's should be trivial; they're all public (except for MPICH-MX). * The MPI installs should all build the most optimal version of the MPI possible (e.g., see OMPI and MPICH2's MPI Install sections). * Note that there's some "weird" stuff for MPICH2 and Intel MPI. See the comments in the ini file for explanations. * If you're not using SLURM, you'll need before_any_exec / after_all_exec sections like Intel MPI's MPI Details for MPICH2 and MVAPICH2. Also note the setenv in Intel MPI's MPI Install section -- I don't know offhand if that'll work for vanilla MPICH2 or whether that was something the Intel MPI team added to mpd. Basically, we want to see if the organizations can take this template and run with it to get performance results back to the MTT database (even with just 2 MPI processes). Let me know if you have any questions / problems. -- Jeff Squyres Cisco Systems
[MTT devel] Thoughts on tagging...
I volunteered to do this on the call today. Here's my thoughts on tagging: 1. From the client, it would be nice to be able to specify a comma- delimited list of tags at any phase. Tags would be inherited by successive phases if not explicitly overridden. E.g., if you specify a "foo" tag in an MPI get, it'll be used in all phases that use that MPI get. Tags can be specified in one of three forms: +foo: means to *add* this tag to the existing/inherited set -foo: means to *remove* this tag from the existing/inherited set foo: if any tag does not have a +/- prefix, then the inherited set is cleared, effectively making the current set of tags be only the non-prefixed tags and +tags For example: [MPI Get: AAA] # + and - have little meaning for MPI Get tags = foo, bar, baz [Test Get: BBB] # + and - have little meaning for Test Get tags = yar, fweezle, bozzle [Test Build: CCC] # Test build inherits tags from MPI Get and Test Get tags = +fa-schizzle, -yar # Resulting tag set: foo, bar, baz, fweezle, bozzle, fa-schizzle [Test build: DDD] # Override everything tags = yowza, gurple # Resulting tag set: yowza, gurple 2. For the reporter, I think we only want authenticated users to be able to create / manipulate tags. Authentication can be via SVN username / password or the HTTPS submit username / password; I don't have strong preferences. Anyone can query on tags, of course. 3. We should have easy "add these results to a tag" and "remove these results from a tag" operations, similar to GMail/labels. I think the rule should be that if you can show MPI details (i.e., not the summary page), you can add/remove tags. Perhaps something as simple as a text box with two buttons: Add tag, Remove tag. 3a. Example: you drill down to a set of test runs. You type in "jeff results" in the text box and click the "add tag" button. This adds the tag "jeff results" to all the result rows that are checked (it is not an error if the "jeff results" tag already exists on some/all of the result rows). 3b. Example: you drill down to a set of test runs. You type in "jeff results" in the text box and click on the "remove tag" button. This removes the tag "jeff results" from all the result rows that are checked (it is not an error if the jeff results" tag is not on some/ all of the result rows). 4. Per Gmail index label listing, it would be nice to see a list of tags that exist on a given result row. It could be as simple as adding another show/hide column for the tags on a given result row. But it gets a little more complicated because one row many represent many different results -- do we show the union of tags for all the rollup rows? Maybe we can use different colors / attributes to represent "this tag exists on *some* of the results in this row" vs. "this tag exists on *all* of the results in this row"...? 4a. If the tags are listed as a column, they should also (of course) be clickable so that if you click on them, you get the entire set of results associated with that tag. 4b. For every tag on a rollup row, it would be good to be able to say "apply this tag to every result in this rollup row" (i.e., this tag had previously only applied to *some* of the results in this rollup row). This could be displayed as a little "+" icon next to the tag name, or somesuch. 4c. Similarly, for every tag, it would be good to have a "remove this tag from every result in this row". This could be displayed as a little "-" icon next to the tag name, or somesuch. 4d. Care would need to be taken to ensure that users would not accidentally click on "+" or "-" icons next to tag names, however. 5. There should also be a simple way to: - see all available tags (perhaps including some kind of indication of how many results that tag represents) - completely delete a tag from the entire database 6. Tags may span multiple phases (install, build, run). If you click on a tag that contains results on all three phases, what should happen? I think it should be context-sensitive: - If you are in a summary environment, you get a summary table showing all relevant results. - If you are in a single phase environment, you see only the results in that phase (perhaps with a clickable icon to see the entire summary table with all the tag's results). 7. Lots of things can, by default, become tags. E.g., org name and platform name can become default tags. I.e., results that are submitted will automatically have the org name and platform name added to the results as tags. -- Jeff Squyres Cisco Systems
