Author: pekka.klarck
Date: Tue Jun 23 12:54:23 2009
New Revision: 2047
Modified:
wiki/HowToWriteGoodTestCases.wiki
Log:
i dont like this wiki format...
Modified: wiki/HowToWriteGoodTestCases.wiki
==============================================================================
--- wiki/HowToWriteGoodTestCases.wiki (original)
+++ wiki/HowToWriteGoodTestCases.wiki Tue Jun 23 12:54:23 2009
@@ -4,120 +4,119 @@
= Introduction =
- * How to write tests in high-level, not how to interact with the SUT
- * Most important guideline is keeping test cases as easy to read as
possible.
+ * How to write tests in high-level, not how to interact with the SUT
+ * Most important guideline is keeping test cases as easy to read as
possible.
= Naming =
== Test suite names ==
- * Names should be as describing as possible
- * Can be relatively long but over 40 characters starts to be too much
+ * Names should be as describing as possible
+ * Can be relatively long but over 40 characters starts to be too much
== Test case names ==
- * Should be describing similarly as test suite names
- * If a suite contain many similar tests, their names can be shorter
+ * Should be describing similarly as test suite names
+ * If a suite contain many similar tests, their names can be shorter
== Keyword names ==
- * Should be describing and clear
- * Should explain what a keyword does, not how it does it
- * Very different abstraction levels
+ * Should be describing and clear
+ * Should explain what a keyword does, not how it does it
+ * Very different abstraction levels
== Naming setup and teardown ==
- * Try to use name that describes what is done
- * Possibly an existing keyword
- * More abstract names acceptable if setup/teardown contain unrelatd steps
- * Listing steps in name is dublication and a maintenance problem
- (e.g. `Login to system, add user, activate alarms and check
balance`)
- * May need to use something generic like `Initialize system`
- * Everyone working with these tests should always understand the name
+ * Try to use name that describes what is done
+ * Possibly an existing keyword
+ * More abstract names acceptable if setup/teardown contain unrelatd steps
+ * Listing steps in name is dublication and a maintenance problem (e.g.
`Login to system, add user, activate alarms and check balance`)
+ * May need to use something generic like `Initialize system`
+ * Everyone working with these tests should always understand the name
= Documentation =
== Test suite documentation ==
- * Often a good idea to add documentation to suites containing tests
+ * Often a good idea to add documentation to suites containing tests
* Higher level suites don't need documentation that often
- * Should contain background information, why tests are created, etc.
- * Don't just repeat test suite name
+ * Should contain background information, why tests are created, etc.
+ * Don't just repeat test suite name
* Better to not have at all if not needed
- * Don't include too much details about test cases
+ * Don't include too much details about test cases
* Tests should be clear enough to
* Dublicate information is waste and maintenance problem
- * Can contain links to more information
+ * Can contain links to more information
== Test case documentation ==
- * Test cases normally don't need documentation
+ * Test cases normally don't need documentation
* Suite's name and documentation and test's name should give enough
background information
* Test cases' structure should be clear enough without documentation
or other comments
- * Sometimes useful so don't be afraid to use it
+ * Sometimes useful so don't be afraid to use it
== User keyword documentation ==
- * Normally not needed
+ * Normally not needed
* Good name and clear structure should be enough
- * Can be used to explain arguments and return values
- * Visible in resource file documentations generated with libdoc.py
+ * Can be used to explain arguments and return values
+ * Visible in resource file documentations generated with libdoc.py
= Test case structure =
- * Test case should be easy to understand
- * One test case should be testing one thing
+ * Test case should be easy to understand
+ * One test case should be testing one thing
* 'Things' can be small (part of single feature) or large (end-to-end)
- * Select suitable abstraction level
+ * Select suitable abstraction level
* Only include information that is relevant for the test case
- * Two kinds of test cases
+ * Two kinds of test cases
* Workflow tests
* Data-driven tests
== Workflow tests ==
- * Steps describe what test does
+ * Steps describe what test does
* Use clear keyword names and suitable abstraction level
* Should contain enough information to run manually
* Should never need documentation or commenting to explain what test
does
- * Can have different abstraction level
+ * Can have different abstraction level
* Tests for a detailed functionality are more precise
* End-to-end tests can be on very high level
* One test should use only one abstraction level
- * Different styles
+ * Different styles
* More techical tests for lower level details and integration tests
* "Executable specifications" act as requiremens
* Everyone (including customer/product owner) should always understand
- * No complex logic
+ * No complex logic
* No for loops or if/else constructs
* Use variable assignements with care
* Test cases should not look like scripts
- * Max 10 steps, preferably less
+ * Max 10 steps, preferably less
== Data-driven tests ==
- * One high-level keyword per test
+ * One high-level keyword per test
* Different arguments create different tests
* The keyword often contains similar workflow as workflow tests
and is created in the same test case file
- * Possible, and recommended, to name column headings
- * Better to have separate tests than test multiple things in one test
+ * Possible, and recommended, to name column headings
+ * Better to have separate tests than test multiple things in one test
* Easier to understand than for example looping
* No tricks needed to go through all test even if some fails
* There's not that much extra work or other overhead
- * If lots of test needed, consider generating them based on a separate
model
+ * If lots of test needed, consider generating them based on a separate
model
= User keywords =
- * Should be easy to understand
+ * Should be easy to understand
* Same rules as with workflow tests
- * Different abstraction levels
- * Can contain some programming logic (for loops, if/else)
+ * Different abstraction levels
+ * Can contain some programming logic (for loops, if/else)
* Especially on lower level keywords
* Complex logic in libraries rather than in user keywords
@@ -126,24 +125,24 @@
== Variable naming ==
- * Clear but not too long names
- * Can use comments in variable table to document them more
- * Use case consistently
+ * Clear but not too long names
+ * Can use comments in variable table to document them more
+ * Use case consistently
* Lower case with variables only available inside certain test or
keyword
* Upper case with others (global, suite or test level)
* Both space and underscore can be used as word separator
- * Recommended to list also variables that are set dynamically in the
+ * Recommended to list also variables that are set dynamically in the
variable table
* Set typically using `Set Global/Suite/Test Variable` keywords
* The initial value should explain where/how the real value is set
== Passing and returning values ==
- * Common approach is to return values from keywords, assign them to
variables
+ * Common approach is to return values from keywords, assign them to
variables
and then pass them as arguments to other keywords
* Clear and easy to follow approach
* Looks like programming
- * Alternative approach is using `Set Test Variable` keyword
+ * Alternative approach is using `Set Test Variable` keyword
* No need to have any programming style in test case level
* More complex to follow, reusing keywords harder
* Avoid below test case level
@@ -151,12 +150,12 @@
= Avoid sleeping =
- * Sleeping is very fragile
- * Safety margins cause too long sleeps on average
- * Instead of sleep, use keyword that polls has certain action occurred
+ * Sleeping is very fragile
+ * Safety margins cause too long sleeps on average
+ * Instead of sleep, use keyword that polls has certain action occurred
* Should have a maximum time to wait
* Keyword names often starts with `Wait Until ...`
* Possible to wrap other keywords inside built-in keyword
`Wait Until Keyword Succeeds`
- * Sometimes sleeping is easiest but use with care
+ * Sometimes sleeping is easiest but use with care
* Never use in keywords that are used often by tests or other keywords
