Revision: 3245
Author: pekka.klarck
Date: Mon May 10 03:29:01 2010
Log: moved the new test execution section to higher level (i.e. moved it to
own file)
http://code.google.com/p/robotframework/source/detail?r=3245
Added:
/trunk/doc/userguide/src/ExecutingTestCases/TestExecution.txt
Modified:
/trunk/doc/userguide/src/ExecutingTestCases/BasicUsage.txt
/trunk/doc/userguide/src/RobotFrameworkUserGuide.txt
=======================================
--- /dev/null
+++ /trunk/doc/userguide/src/ExecutingTestCases/TestExecution.txt Mon May
10 03:29:01 2010
@@ -0,0 +1,148 @@
+Test execution
+--------------
+
+This section describes how the test suite structure created from the parsed
+test data is executed, how to continue executing a test case after
failures,
+and how to stop the whole test execution gracefully.
+
+.. contents::
+ :depth: 2
+ :local:
+
+Test execution flow
+~~~~~~~~~~~~~~~~~~~
+
+Test execution starts with highest level test suite's setup if one exists.
+After setup is executed, sub suites (in case of directory suite) or test
cases
+(in case of file suite) are executed in order specified in `test execution
order`_
+chapter. In case of directory suite, it's sub suites are executed similar
manner.
+
+In case suite setup fails, sub suites and/or tests are executed, but failed
+immediately with information about failure in suite setup.
+
+Test case execution is started by executing setup and continued by
executing
+keywords one at a time. In case failure occurs during setup or keyword
execution,
+test execution is stopped unless failure type is `continue on failure`__.
Teardown
+is executed always as last step even test execution fails.
+
+__ `continue on failure`_
+
+Starting from Robot Framework 2.5 all keywords in suite and test teardowns
are
+executed, on `continue on failure mode`__, even if failures occur. If
teardown
+execution is `stopped by user or with special exception`__, teardown
execution is
+stopped.
+
+__ `continue on failure`_
+__ `stopping test execution gracefully`_
+
+Following structure demonstrates the test execution order:
+
+- suite setup
+
+ - suite setup
+
+ - suite setup
+
+ - test setup
+
+ - test keywords
+
+ - test teardown
+
+ - test setup
+
+ - test keywords
+
+ - test teardown
+
+ - suite teardown
+
+ - suite setup
+
+ - test setup
+
+ - test keywords
+
+ - test teardown
+
+ - test setup
+
+ - test keywords
+
+ - test teardown
+
+ - suite teardown
+
+ - suite teardown
+
+- suite teardown
+
+
+Test execution order
+~~~~~~~~~~~~~~~~~~~~
+
+Test cases in a test suite are executed in the order they are specified
+in the test case file where they are created. Test suites inside a higher
level
+test suite are executed in case-insensitive alphabetical order based on
the file
+or directory name. Finally, if multiple files and/or directories are given
from
+the command line, they are executed in the order they are specified.
+
+If there is a need to use certain test suite execution order inside a
+directory, it is possible to add prefixes like :path:`01` and
+:path:`02` into file and directory names. Starting from Robot
+Framework 2.1, such prefixes are automatically removed if they are
+separated from the base name with two underscores like in the examples
+below. These prefixes will not be visible anywhere in reports or
+logs::
+
+ 01__my_suite.html -> My Suite
+ 02__another_suite.html -> Another Suite
+
+If the alphabetical ordering of test suites inside suites is problematic,
+a good workaround is giving them separately in the required order. This
easily
+leads to overly long start-up commands, but `argument files`_ allow
listing
+files nicely one file per line. It is also possible to `randomize the
execution
+order of test cases or test suites`__ using :opt:`--runmode` option.
+
+__ `Setting the execution mode`_
+
+
+Continue on failure
+~~~~~~~~~~~~~~~~~~~
+
+Starting from Robot Framework 2.5 test execution could have been continued
even
+case of failure. This can be achieved by using
+`Run Keyword And Continue On Failure` keyword from BuiltIn library or
+implementing own keyword failing with special exception. How to create such
+error is explained in `the test library API section`__. When test execution
+ends, test case is failed with all the continue on failures that have
occurred
+during test execution.
+
+__ `continuing test execution despite of failures`_
+
+Stopping test execution gracefully
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Starting from Robot Framework 2.5 it have been possible to stop test
execution
+gracefully. When test execution is stopped, the remaining tests are marked
+failed and their message will be :msg:`Test execution was stopped due to a
fatal error`.
+
+Test execution can be stopped by pressing :code:`Ctrl-C` in the console in
which
+the test execution was started. In case :prog:`pybot` is used, execution is
+stopped immediately and all the remaining tests will fail. In
case :prog:`jybot`
+is used, the test execution ends immediately when currently running
keyword returns.
+This limitation is caused by a bug in Jython's signal handling.
Second :code:`Ctrl-C`
+will forcefully stop the test execution.
+
+When test are executed in background process, :code:`INT` or :code:`TERM`
+signals can be used for stopping the test execution using tools like
+:prog:`kill`. Second signal can be used to forcefully stop the test
execution.
+Same limitation with :prog:`jybot` applies as with stopping
with :code:`Ctrl-C`.
+As signaling is not supported on Windows, there is no easy way to use this
+feature on it.
+
+Another possibility to stop test execution is having a keyword fail with a
+special error. How to create such error is explained in
+`the test library API section`__.
+
+__ `Stopping test execution`_
=======================================
--- /trunk/doc/userguide/src/ExecutingTestCases/BasicUsage.txt Wed May 5
14:39:48 2010
+++ /trunk/doc/userguide/src/ExecutingTestCases/BasicUsage.txt Mon May 10
03:29:01 2010
@@ -10,7 +10,6 @@
:depth: 2
:local:
-
Starting test execution
~~~~~~~~~~~~~~~~~~~~~~~
@@ -23,7 +22,6 @@
jybot [options] datasources
interpreter /path/to/robot/runner.py [options] datasources
-
Different runner scripts
''''''''''''''''''''''''
@@ -50,7 +48,6 @@
options can be used to alter the test execution or generated
outputs in some way.
-
Specifying test data to be executed
'''''''''''''''''''''''''''''''''''
@@ -144,7 +141,6 @@
variables. If the options that take only one value are used several
times, the value given last is effective.
-
Option value as simple pattern
``````````````````````````````
@@ -156,7 +152,6 @@
:cli:`--include a???` matches any tag that is four characters long and
starts with a character :code:`a`.
-
Generated outputs
'''''''''''''''''
@@ -189,7 +184,6 @@
detailed log file. These files and other possible output files are
discussed in more detail in the section `Different output files`_.
-
Return codes
''''''''''''
@@ -221,7 +215,6 @@
variable. If you use some external tool for running tests, consult its
documentation for how to get the return code.
-
Errors and warnings during execution
''''''''''''''''''''''''''''''''''''
@@ -254,149 +247,6 @@
__ `Deprecating keywords`_
__ `Logging information`_
-
-Test execution
-~~~~~~~~~~~~~~
-
-Test execution flow
-''''''''''''''''''''
-
-Test execution starts with highest level test suite's setup if one exists.
-After setup is executed, sub suites (in case of directory suite) or test
cases
-(in case of file suite) are executed in order specified in `test execution
order`_
-chapter. In case of directory suite, it's sub suites are executed similar
manner.
-
-In case suite setup fails, sub suites and/or tests are executed, but failed
-immediately with information about failure in suite setup.
-
-Test case execution is started by executing setup and continued by
executing
-keywords one at a time. In case failure occurs during setup or keyword
execution,
-test execution is stopped unless failure type is `continue on failure`__.
Teardown
-is executed always as last step even test execution fails.
-
-__ `continue on failure`_
-
-Starting from Robot Framework 2.5 all keywords in suite and test teardowns
are
-executed, on `continue on failure mode`__, even if failures occur. If
teardown
-execution is `stopped by user or with special exception`__, teardown
execution is
-stopped.
-
-__ `continue on failure`_
-__ `stopping test execution gracefully`_
-
-Following structure demonstrates the test execution order:
-
-- suite setup
-
- - suite setup
-
- - suite setup
-
- - test setup
-
- - test keywords
-
- - test teardown
-
- - test setup
-
- - test keywords
-
- - test teardown
-
- - suite teardown
-
- - suite setup
-
- - test setup
-
- - test keywords
-
- - test teardown
-
- - test setup
-
- - test keywords
-
- - test teardown
-
- - suite teardown
-
- - suite teardown
-
-- suite teardown
-
-
-Test execution order
-''''''''''''''''''''
-
-Test cases in a test suite are executed in the order they are specified
-in the test case file where they are created. Test suites inside a higher
level
-test suite are executed in case-insensitive alphabetical order based on
the file
-or directory name. Finally, if multiple files and/or directories are given
from
-the command line, they are executed in the order they are specified.
-
-If there is a need to use certain test suite execution order inside a
-directory, it is possible to add prefixes like :path:`01` and
-:path:`02` into file and directory names. Starting from Robot
-Framework 2.1, such prefixes are automatically removed if they are
-separated from the base name with two underscores like in the examples
-below. These prefixes will not be visible anywhere in reports or
-logs::
-
- 01__my_suite.html -> My Suite
- 02__another_suite.html -> Another Suite
-
-If the alphabetical ordering of test suites inside suites is problematic,
-a good workaround is giving them separately in the required order. This
easily
-leads to overly long start-up commands, but `argument files`_ allow
listing
-files nicely one file per line. It is also possible to `randomize the
execution
-order of test cases or test suites`__ using :opt:`--runmode` option.
-
-__ `Setting the execution mode`_
-
-
-Continue on failure
-'''''''''''''''''''
-
-Starting from Robot Framework 2.5 test execution could have been continued
even
-case of failure. This can be achieved by using
-`Run Keyword And Continue On Failure` keyword from BuiltIn library or
-implementing own keyword failing with special exception. How to create such
-error is explained in `the test library API section`__. When test execution
-ends, test case is failed with all the continue on failures that have
occurred
-during test execution.
-
-__ `continuing test execution despite of failures`_
-
-Stopping test execution gracefully
-''''''''''''''''''''''''''''''''''
-
-Starting from Robot Framework 2.5 it have been possible to stop test
execution
-gracefully. When test execution is stopped, the remaining tests are marked
-failed and their message will be :msg:`Test execution was stopped due to a
fatal error`.
-
-Test execution can be stopped by pressing :code:`Ctrl-C` in the console in
which
-the test execution was started. In case :prog:`pybot` is used, execution is
-stopped immediately and all the remaining tests will fail. In
case :prog:`jybot`
-is used, the test execution ends immediately when currently running
keyword returns.
-This limitation is caused by a bug in Jython's signal handling.
Second :code:`Ctrl-C`
-will forcefully stop the test execution.
-
-When test are executed in background process, :code:`INT` or :code:`TERM`
-signals can be used for stopping the test execution using tools like
-:prog:`kill`. Second signal can be used to forcefully stop the test
execution.
-Same limitation with :prog:`jybot` applies as with stopping
with :code:`Ctrl-C`.
-As signaling is not supported on Windows, there is no easy way to use this
-feature on it.
-
-Another possibility to stop test execution is having a keyword fail with a
-special error. How to create such error is explained in
-`the test library API section`__.
-
-__ `Stopping test execution`_
-
-
Post-processing outputs
~~~~~~~~~~~~~~~~~~~~~~~
@@ -412,7 +262,6 @@
that arguments to :prog:`rebot` are XML output files instead of test
data files or directories.
-
Creating different reports and logs
'''''''''''''''''''''''''''''''''''
@@ -445,7 +294,6 @@
__ `Modifying Java startup parameters`_
-
Combining outputs
'''''''''''''''''
@@ -496,7 +344,6 @@
C:\>rebot --version
Rebot 2.0.3 (Python 2.5.2 on win32)
-
Escaping complicated characters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -554,7 +401,6 @@
data, are escaped. Escape character sequences thus need to be
selected carefully.
-
Argument files
~~~~~~~~~~~~~~
@@ -602,7 +448,6 @@
__ `Specifying test data to be executed`_
-
Creating start-up scripts
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -692,7 +537,6 @@
set CP=%CP%;%1
goto :eof
-
Modifying Java startup parameters
'''''''''''''''''''''''''''''''''
=======================================
--- /trunk/doc/userguide/src/RobotFrameworkUserGuide.txt Tue Oct 20
06:20:04 2009
+++ /trunk/doc/userguide/src/RobotFrameworkUserGuide.txt Mon May 10
03:29:01 2010
@@ -57,6 +57,7 @@
:local:
.. include:: ExecutingTestCases/BasicUsage.txt
+.. include:: ExecutingTestCases/TestExecution.txt
.. include:: ExecutingTestCases/ConfiguringExecution.txt
.. include:: ExecutingTestCases/OutputFiles.txt
