Hello Ethan,
On Mon, Apr 13, 2009 at 5:44 PM, Ethan Mallove <ethan.mall...@sun.com>wrote: > > Will this translate to something like > lib/MTT/Reporter/GoogleDatabase.pm? If we are to move away from the > current MTT Postgres database, we want to be able to submit results to > both the current MTT database and the new Google database during the > transition period. Having a GoogleDatabase.pm would make this easier. > I think we should keep both storage options: current postgress and datastore. The mtt changes will be minor to support datastore. Due that fact that google appengine API (as well as datastore API) can be python or java only, we will create external scripts to manipulate datastore objects: The mtt will dump test results in xml format. Then, we provide two python (or java?) scripts: mtt-results-submit-to-datastore.py - script will be called at the end of mtt run and will read xml files, create objects and save to datastore mtt-results-query.py - sample script to query datastore and generate some simple visual/tabular reports. It will serve as tutorial for howto access mtt data from scripts for reporting. Later, we add another script to replace php web frontend. It will be hosted on google appengine machines and will provide web viewer for mtt results. (same way like index.php does today) > > > > > b. mtt_save_to_db.py - script which will go over mtt scratch dir, > find > > all xml files generated for every mtt phase, parse it and save to > > datastore, preserving test results relations,i.e. all test results > will > > be grouped by mtt general info: mpi version, name, date, .... > > > > c. same script can scan, parse and save from xml files generated by > > wrapper scripts for non mtt based executions (fluent, ..) > > > > I'm confused here. Can't MTT be outfitted to report results of a > Fluent run? > I think we can enhance mtt to be not only mpi testing platform, but also to serve as mpi benchmarking platform. We can use datastore to keep mpi-based benchmarking results in the same manner like mtt does for testing results. (no changes to mtt required for that, it is just a side effect of using datastore to keep data of any type) > > > > d. mtt_query_db.py script will be provided with basic query > capabilities > > over proposed datastore object model. Most users will prefer writing > > custom sql-like select queries for fetching results. > > > > 3. Important notes: > > ============== > > > > a. The single mtt client execution generates many result files, > every > > generated file represents test phase. This file contains test > results > > and can be characterized as a set of attributes with its values. > Every > > test phase has its own attributes which are differ for different > phases. > > For example: attributes for TestBuild phase has keys "compiler_name, > > compiler_version", the MPIInstall phase has attributes: prefix_dir, > > arch, .... > > Hence, most of the datastore objects representing phases of MTT* are > > derived from "db.Expando" model, which allows having dynamic > attributes > > for its derived sub-classes. > > > > The attached is archive with a simple test for using datastore for > mtt. > > Please see models.py file with proposed object model and comment. > > > > I don't see the models.py attachment. > I just sent original email with attachment, tell me if you want me to send it again. > > regards Mike
