cool - this is going straight on my reading pile! Before I read it, may I ask if this is appropriate material to convert to xml and move onto the website? It would get better visibility there IMHO.
Regards, Tim [EMAIL PROTECTED] wrote: > Author: varlax > Date: Thu Aug 2 03:07:45 2007 > New Revision: 562065 > > URL: http://svn.apache.org/viewvc?view=rev&rev=562065 > Log: > Additional document for HARMONY-4293 > > Added: > harmony/enhanced/buildtest/branches/2.0/dev-guide.txt (with props) > > Added: harmony/enhanced/buildtest/branches/2.0/dev-guide.txt > URL: > http://svn.apache.org/viewvc/harmony/enhanced/buildtest/branches/2.0/dev-guide.txt?view=auto&rev=562065 > ============================================================================== > --- harmony/enhanced/buildtest/branches/2.0/dev-guide.txt (added) > +++ harmony/enhanced/buildtest/branches/2.0/dev-guide.txt Thu Aug 2 03:07:45 > 2007 > @@ -0,0 +1,591 @@ > +============================================================================== > + BTI Design Document > +============================================================================== > +:Title: BTI Design Document - a Guide for BTI Developers > +:Author: Alexander Kleymenov <kleymenov{ at }gmail{ . }com> > +:Date: 2007/07/05 > + > +This document contains detailed description of each of the functional parts > +of Harmony BTI. It could be used as a *Guide for BTI Developers*. > +For quick start with BTI please look at `README`_ file. > + > +Architectural parts of BTI covered in this document can be summarized > +as follows: > + > +:`Execution Core`_: > + Maintains BTI configuration and Tests execution. > + Acts as a glue for other components of BTI. > + Provides some auxiliary functionality for implementation of > + test suites adaptors. > + > +:`Test Suite Adaptors`_: > + The Ant scripts integrating the test suites to BTI by means of > + implementation *adapting* interface. > + Consist of ``adaptor.xml`` file maintaining test execution > + and ``parameters.xml`` file describing configuration parameters. > + > +:`CTE Module`_: > + Performs Continuous Test Execution. > + Is built on the base of CruiseControl system. > + > +:`Execution Status Notifier`_: > + Sends mail alerts with the status of test execution. > + > +:`External Dependencies Fetching Module`_: > + Provides the unified means for description of external dependencies > + used by test suites or BTI. > + Downloads and installs the external libraries. > + Allows sharing of external libraries between test suites. > + > + > +Execution Core > +============== > + > +Implemented by the following source files:: > + > + ./build.xml > + ./buildtest.bat > + ./buildtest.sh > + ./scripts/exec-adaptor.xml > + ./scripts/framework.xml > + ./scripts/main.xml > + > +This part of BTI is responsible for BTI configuration and execution. > +It acts as an execution engine gluing all other functional parts of > +BTI together. It also provides some functionality supporting the development > +of Test Suite Adaptors. > + > + > +BTI Configuration > +----------------- > + > +The BTI configuration is made by means of the property files. > +The first properties file:: > + > + ./scripts/framework.properties > + > +contains the properties affecting internals of BTI and *SHOULD NOT* > +be modified by user. The second file:: > + > + ./scripts/parameters.xml > + > +is an XML Ant properties file containing the configuration parameters for all > +of the functional parts of BTI. These parameters can be configured by user > +through the third configuration file:: > + > + ./framework.local.properties > + > +By means of this file user can also tune some test suites' specific > parameters > +defined in:: > + > + ./adaptors/*/parameters.xml > + > +files. > + > +The last configuration file is:: > + > + ./required-parameters.properties > + > +It is generated by BTI and contains the assignments of parameters *required* > by > +test suites selected for execution (more on this in `Test Suite Adaptors`_ > +section). > + > + > +BTI Execution > +------------- > + > +BTI execution is launched by means of a command line UI provided by the Core. > +Currently this UI consists of the following Ant targets expected from user as > +parameters to BTI command line scripts (``buildtest.bat`` for MS Windows[x]_, > +and ``buildtest.sh`` for Linux): > + > +``install`` > + Does an installation of BTI on local workstation from remote SVN > + repository. > + Checks out (for initial execution) or updates (for consequent ones) > + BTI implementation files (``./scripts`` directory) and test suite > + adaptors (``./adaptors`` directory). > + This target *MUST* be run at first after initial BTI checkout (see > + ``./README.txt`` for how to get BTI from SVN repository). > + Generates ``./framework.local.properties`` property file for local BTI > + configuration. > + > +``framework-update`` > + Does an update of current BTI installation from the SVN repository. > + Updates BTI implementation scripts and test suite adaptors. > + > +``setup`` > + Does a setup of the test run. Expects the ``test.suites`` property > + to be set to comma separated list of test suites selected for execution. > + Produces 3 files: > + > + ``./required-parameters.properties`` > + property file containing the assignments of parameters required > + by selected test suites. > + > + ``./selected-suites.properties`` > + property file containing the test run configuration. Describes the > + order in which the selected suites will be run, and dependencies > + between them. > + > + ``./resources.location`` > + property file describing the locations of external resources needed > by > + configured test run. > + > +``run`` > + Launches test execution. The selected test suites will be executed in > + order defined by ``test.suites.names.list`` property in > + ``./selected-suites.properties`` file. > + At first, BTI executes ``setup`` targets of adaptor scripts of all > + of the selected test suites (in defined order). If ``setup`` fails for > + some of the suite, this suite won't be ``run``, and all dependent test > + suites won't be ``setup`` and ``run``. BTI will warn about all of > + the failed and skipped test suites by means of its log messages. > + At second, after successful ``setup``, BTI launches ``run`` targets of > + selected test suites. Similarly to ``setup`` BTI will skip the execution > + of dependent test suites if their superior test suites failed to > + ``run``. The information about failed and skipped test suites will be > + printed out by BTI to its log. > + After all runs of the selected test suites BTI ends with *SUCCESS* if > + there were no test suites failures, and *ERROR* if some suites could not > + be set up or failed to run. > + > +``run-cc`` > + Does the same as ``run``, but test cycle is continuously repeated. Each > + iteration of test suites execution is triggered whether by > + modifications happened in some of the controlled by test suites SVN > + repositories, or by schedule provided by user. See `CTE Module`_ for > more > + information on Continuous Test Execution. > + > +``clean`` > + Consequently launches ``clean`` targets of adaptors of selected test > + suites. > + > +``test-mail`` > + Sends test mail notification. Used to check the operability of the > + current configurations of `Execution Status Notifier`_ module. > + > +The execution of BTI can be configured by the following properties specified > +in ``./framework.local.properties`` file:: > + > + framework.parameters.optional.dolog > + framework.parameters.repository.noupdate > + framework.parameters.repository.awaited > + > +For descriptions of these parameters look in ``./scripts/parameters.xml`` > file. > + > + > + > +Test Suite Adaptors > +=================== > + > +A Test Suite Adaptor consists of two Ant files placed under > +``./adaptors/${suite.name}`` directory: > + > +adaptor.xml > +----------- > + > +Regular Ant project file implementing 3 targets: ``setup``, ``run``, and > +``clean``: > + > + **setup** > + has to perform all the works on setting of the test suite on the > + local workstation: it should download and build the sources for test > + suite, check required prerequisites, and so on - everything that > + should be done before first run of the test suite on the workstation. > + `Execution Core`_ provides some functionality which can be reused > + here: > + > + ``<svn-co url="${url}" trunk="${trunk}"/>`` - macrodefinition > checking > + out the SVN repository located at ``${url}`` to directory pointed by > + ``${trunk}``. > + > + ``<svn-update-bt path="${path}"/>`` - macrodefinition updating the > + local ``${path}`` of BTI repository. It is useful when the sources of > + the test suite are located under BTI's SVN repository. > + > + **run** > + performs the actual run of the test suite. It should "``<fail>``" > + if test suite's execution status is FAILED. > + > + **clean** > + cleans the products of test suite's execution > + > +The following properties provided by `Execution Core`_ can be useful for > +implementing these targets: > + > + :``suite.name``: > + holding the name of the suite > + > + :``suite.adaptor.dir``: > + the directory where adaptor is placed > + > + :``test.sources.dir``: > + the root directory containing the test sources. > + Usually the test sources for each particular test suite are placed > + under ``${test.sources.dir}/${suite.name}`` directory. > + > + :``temp.files.dir``: > + the directory for temporal files > + > + :``checkouts.dir``: > + the directory for external checkouts. For example *classlib* test > + suite checks out its sources to ``${checkouts.dir}/classlib`` > directory. > + > + :``results.dir``: > + the BTI's directory for storing the test execution results. Usually > + test suite uses ``${results.dir}/${suite.name}`` directory for this > + purpose. > + > + :``classes.dir``: > + the directory for compiled java classes. It's needed if the test > suite > + adaptor uses some java classes which should be compiled. > + > +parameters.xml > +-------------- > + > +Ant's xml property file containing declarations of parameters describing > +the test suite. This xml property file is loaded after > +``adaptor.xml`` file and, thus, can contain references to the properties > +defined there. The parameters described in parameters.xml file can be > divided > +into two categories: *execution* and *integration* parameters. > + > +Execution Parameters > +~~~~~~~~~~~~~~~~~~~~ > + > +are the parameters affecting the test suite execution. > +These parameters includes: > + > +*depends* > ++++++++++ > + > + contains comma separated list of all of the test suites on which this > test > + suite depends. Actually this parameter defines the order in which the > + test suite will be executed: it will be executed after suites defined by > + this parameter. If some of the listed test suites were not selected for > + execution, they are removed from this parameter. If default value of this > + parameter does not correspond to selected set of test suites, User > *SHOULD* > + change this parameter by means of ``${suite.name}.parameters.depends`` > + property specified in ``./framework.local.properties`` file. > + > +*shared* parameters > ++++++++++++++++++++ > + > + are the parameters holding the values which can be used by other test > + suites. Such parameters affecting the execution of other test suites. > + The path on built product (as JVM for example) can be an example of such > + parameter. > + > +*required* parameters > ++++++++++++++++++++++ > + > + are the parameters for which the test suite is not able to provide the > + default values: the values for these parameters can be specified only > + from the outside of the test suite. For example: such parameters can > + hold the path to tested JVM - test suite can't guess this information, > + so it can't provide the default value for this parameter. On the BTI's > + ``setup`` stage, all the required parameters for the selected > + test suites are collected in ``./required-parameters.properties`` file > and > + if some of such parameters has no value, BTI stops with *ERROR* message. > + > + **Note:** the declaration of *required* parameter should have a > + ``description`` attribute. Without this attribute such a > + parameter won't be regarded as *required*. > + > + While test suite itself can not provide the default value for such a > + parameter, it is possible to provide it from BTI's (read "external") > + side. It is done by means of connecting *required* parameter of test > + suite with *shared* parameter of test suite which can provide *required* > + value. Such test suites providing their *shared* parameters should be > + listed in ``depends`` parameter of dependent test suite - it will allow > + them to be executed before dependent test suite, and, correspondingly, > + to build a product pointed by the value of *shared* parameter before > + dependent test suite will use it. > + The values for *required* parameters can be specified in three places: > + > + * test suites ``./parameters.xml`` file can contain only connections > + between *required* parameters and *shared* parameters of some other > + test suites. These test suites should be listed in ``depends`` > parameter. > + > + **Note:** if these test suites are not selected for execution, these > + *shared* parameters won't be resolved and *required* parameter won't > + have the value. BTI will stop with ERROR message requesting to specify > + the value for this unset *required* parameters. > + > + * ``./framework.local.properties`` file can contain connections between > + the *required* and *shared* parameters, as well as direct evaluation of > + *required* parameters with some value. The value provided in this file > + takes precedence over the value in ``./parameters.xml`` file. > + > + * ``./required-parameters.properties`` file can contain only direct > + evaluation of required parameter with some value. It can't contain any > + property resolutions. The value provided in this file takes precedence > + on the values provided in ``./framework.local.properties`` and > + ``./parameters.xml`` files. > + > + **Note:** the *required* parameters in this file > + appears in short format: ``${suite.mane}.${required.parameter.name}``. > For > + example, if *required* parameter has been defined in > ``parameters.xml`` as > + ``suite.parameters.required.some.param``, in > + ``./required-parameters.properties`` it will be in the following form: > + ``suite.some.param``. > + > +*optional* parameters > ++++++++++++++++++++++ > + > + are the parameters allowing to user to influence how the test suite > + will be executed. Such parameters have some default values and should > + have the description. > + > + > +Integration Parameters > +~~~~~~~~~~~~~~~~~~~~~~ > + > +describe the support from BTI's side needed for correct test suite execution. > +These parameters includes: > + > +*external* parameters > ++++++++++++++++++++++ > + > + are the parameters describing the dependencies of the test suite on > + external resources located at some point of the web. This parameters > are > + addressed to the `External Dependencies Fetching Module`_ which is > + responsible for downloading and sharing them between the test suites. > + > +*cc* parameters > ++++++++++++++++ > + > + are the parameters describing test suite's specific of continuous > + execution. These parameters describe files and SVN repositories > monitored > + by test suite, define bootstrapping and publishing procedures, do some > + test suite's specific configurations of `Execution Status Notifier`_ > + module. For more information see `CTE Module`_ section. > + > +*mail* parameters > ++++++++++++++++++ > + > + are the parameters describing the test suite's specific configuration > of > + `Execution Status Notifier`_ module. Initially these parameters were > + *cc* parameters, but now they have been moved into special category. > + The parameters specified here take precedence over corresponding *cc* > + ones. This parameters allows to specify the summary information to be > + sent with mail notifications, files to be attached, and so on. See > + `Execution Status Notifier`_ for more details. > + > + > + > +CTE Module > +========== > + > +Continuous Test Execution module consists of the following source files:: > + > + ./scripts/cc-project.xml > + ./scripts/cc-exec-adaptor.xml > + ./scripts/templates/cc-config-template.xml > + ./scripts/templates/cc-project-template.xml > + > +This module is responsible for performing of continuous test execution > +launched by ``run-cc`` BTI's target. The work of this module can be divided > into > +2 steps: `preparation`_ and `execution`_. > + > +Preparation > +----------- > + > +On this stage, CTE Module gathers the parameters describing > CruiseControl-related > +configuration and generates CC project files (CC configurations) for each of > +selected test suites. The parameters defining these configurations are > +provided by two sources: > + > +- BTI defines *global* CTE parameters common for all test suites. Such > + parameters include the information about web ports, timeouts, etc. and > + can be (along with their descriptions) seen in ``./scripts/parameters.xml`` > + file under ``<cc>`` section. > + > + **Note:** > + These parameters are *user configurable*. It means that their > + values can be changed by user through the corresponding properties > + specified in ``./framework.local.properties`` file. > + > +- Each of selected test suites uses its own set of CTE parameters > + describing the specific of its continuous execution. These parameters > + appear in test suites' ``parameters.xml`` file under ``<cc>`` section: > + > + :``trigger``: - (no value by default) > + - path to local file which modification will trigger the execution > + of test suite. > + > + :``usesvn``: - (no value by default) > + - the URL of SVN repository which modification will trigger the > + test suite's execution. There can several URLs be set. > + Every other parameter with ``usesvn.`` prefix is considered as URL > + for SVN repository. > + > + :``publish``: - (default is ``true``) > + - determines whether do status notification publishing for this test > + suite or not. The value of this parameter can be changed by user > + through the ``./framework.local.properties`` configuration file. > + In future this parameter will go to ``<mail>`` section of parameters > + configuration file (so it won't be a CTE configuration parameter). > + > + :``results``: - (no value by default) > + - path to directory containing the test execution result files in XML > + JUnit format for status notification message. This parameter will > + go to ``<mail>`` section. > + > + The `Execution Status Notifier`_ configuration parameters may also > + appear in ``<cc>`` section. But their normal place has been moved to > + ``<mail>`` section. They are: ``summary``, ``message``, and ``attachs``. > + They describe summary, content of the notification message and files to be > + attached with the message correspondingly. > + > + **Note:** > + These parameters (except ``publish``) are `Integration Parameters`_ and > + are *not user configurable*. It means that their values should not be > changed > + by user. > + > +Execution > +--------- > + > +After ``setup`` of each of the selected test suites, BTI launches > CruiseControl > +with generated configuration. CC launches test suites in order described by > +``test.suites.names.list`` property defined in > ``./selected-suites.properties`` > +file generated on ``setup`` stage. > +If some of the test suites failed, dependent test suites won't be run. > + > +The execution log of each of the test suites > +can be found in CC's web interface under *build artifacts* section. Web > +interface of CC is available at localhost with > +``framework.parameters.cc.webport`` port number which is 8080 by default. > + > +As stated in `README`_ file, there are two modes of continuous execution: > +SVN modifications triggered and scheduled runs. SVN modifications triggered > runs > +are executed each time monitored SVN of trigger file modifications occur. > +Scheduled runs are launched at the particular time specified by the following > +properties in framework.local.properties file: > + > + ``framework.parameters.schedule.day=Tuesday`` > + Do launch test suites execution on particular day. > + If this property is not defined, the test run will be launched > everyday. > + > + ``framework.parameters.schedule.time=1530`` > + Do launch test suites execution at this particular time. > + Should be defined for scheduled execution mode. > + > + > + > +Execution Status Notifier > +========================= > + > +Implemented by the following source file:: > + > + ./scripts/publisher.xml > + > +This module performs the composition and sending of the message with > +test suite execution status. The work of this module is regulated by means of > +particular configuration parameters defined either by BTI or by > +test suites. The parameters defined by BTI have the global influence and > +take effect for all of the suites. Such the parameters are (look in > +``./scripts/parameters.xml`` file for more info): > + > + :``framework.parameters.usemail``: > + the notification publishing will be in use only if this parameter > is > + set and contains the address of SMTP server to be used for mailing. > + > + :``framework.parameters.usemail.always``: > + defines the notification mode. If it holds ``true`` value - the > + notifications will be sent regardless of execution status. > + But if it holds ``false`` - notifications will be sent only after > + execution status changes (from *PASSED* to *FAILED*, and vice > versa) > + > + :``framework.parameters.usemail.to``: > + should contain the address to where notifications should be sent. > + Use comma for multiple values. > + > + :``framework.parameters.usemail.from``: > + should contain the address from where notifications should be sent > + Use comma for multiple values. > + > + :``framework.parameters.usemail.suppress.exelogs``: > + controls whether execution logs will be sent with failure messages > or > + not. > + > + :``framework.parameters.usemail.subject.prefix``: > + sets up the prefix for notifications subject line. > + > +Also each of the test suite can provide its own parameters configuring the > +status notifications. They are described in test suite's parameters.xml file > +under <mail> section and their values should not be changed by user: > + > + :``summary``: > + the path to file holding test suite's execution summary. This > content > + will be prepended to the notification message > + > + :``message``: > + the path to file holding test suite's message content. This content > + will replace one generated by default. > + > + :``attachs``: > + comma separated paths to files to be attached with notification. > + > +Sometimes it is necessary for test suite to override some global parameter > +defined by framework's parameters. It can be done by means of parameters > +specified in ``./framework.local.properties`` file having > +``${suite.name}.parameters.mail`` prefix instead of > +``framework.parameters.usemail`` one. > + > +For example, if the User needs for *test_suite* to use particular destination > +mail address, the following property can do that: > + > [EMAIL PROTECTED] > + > +In this case status notifications for all of the test suites will be sent to > +the address specified by ``framework.parameters.usemail.to`` parameter, > +but notifications for *test_suite* will be sent to [EMAIL PROTECTED] > + > + > + > +External Dependencies Fetching Module > +===================================== > + > +With this module BTI offers the common way to fetch, use and share external > +libraries between the test suites. It is done by means of parameters > described > +in ``<external>`` section of ``parameters.xml`` files. The description of > the > +external library should contain at least 2 fields: URL from where to take > the > +dependency, and MD5 value to check the dependency. Also there can be some > +attributes describing the type of the dependency and some installation > +information. > + > +BTI on ``setup`` stage fetches all the dependencies according to their > +descriptions and sets up corresponding properties to access them. > +For example: let's look at the following description:: > + > + <external> > + <my-dependency > + url="http://host.of/my-dependency.zip"; > + md5="87A617D7E65979AE01B5ACE4649EB40B" > + dir="my-dep-dir" > + inc="*/lib/*.jar" > + /> > + </external> > + > +It describes dependency called *my-dependency* which is located at specified > +URL and has particular MD5 check sum. During the local installation of > +downloaded archive only jar files from `lib` directory will be unzipped and > +placed under `my-dep-dir` directory. The path to this directory will be > +provided by ``ext.my-dependency.location`` property. > + > +This module is under development and provides only simple functionality. > +For working usage examples, please, look at ``./scripts/parameters.xml`` > file. > + > +------------------------------------------------------------------------------ > + > +.. [x] Other brands and names are the property of their respective owners > +.. _README: ./README.txt > + > +.. Use "Vim reStructured Text" utility to generate HTML: > + http://skawina.eu.org/mikolaj/vst.html > + or online renderer: > + http://www.hosting4u.cz/jbar/rest/rest.html > + > +.. vim:filetype=rst > + > > Propchange: harmony/enhanced/buildtest/branches/2.0/dev-guide.txt > ------------------------------------------------------------------------------ > svn:eol-style = native > > >
