Author: pekka.klarck
Date: Tue Jun 23 12:45:03 2009
New Revision: 2044
Added:
wiki/HowToWriteGoodTestCases.wiki
Log:
initial skeleton, needs more meat and examples
Added: wiki/HowToWriteGoodTestCases.wiki
==============================================================================
--- (empty file)
+++ wiki/HowToWriteGoodTestCases.wiki Tue Jun 23 12:45:03 2009
@@ -0,0 +1,162 @@
+#summary Guidelines for writing good test cases using Robot Framework
+
+<wiki:toc max_depth="2" />
+
+= Introduction =
+
+ * How to write tests in high-level, not how to interact with the SUT
+ * Most important guideline is keeping test cases as easy to read as
possible.
+
+
+= Naming =
+
+== Test suite names ==
+
+ * Names should be as describing as possible
+ * Can be relatively long but over 40 characters starts to be too much
+
+== Test case names ==
+
+ * Should be describing similarly as test suite names
+ * If a suite contain many similar tests, their names can be shorter
+
+== Keyword names ==
+
+ * Should be describing and clear
+ * Should explain what a keyword does, not how it does it
+ * Very different abstraction levels
+
+== Naming setup and teardown ==
+
+ * Try to use name that describes what is done
+ - Possibly an existing keyword
+ * More abstract names acceptable if setup/teardown contain unrelatd steps
+ - Listing steps in name is dublication and a maintenance problem
+ (e.g. `Login to system, add user, activate alarms and check balance`)
+ - May need to use something generic like `Initialize system`
+ * Everyone working with these tests should always understand the name
+
+
+= Documentation =
+
+== Test suite documentation ==
+
+ * Often a good idea to add documentation to suites containing tests
+ - Higher level suites don't need documentation that often
+ * Should contain background information, why tests are created, etc.
+ * Don't just repeat test suite name
+ - Better to not have at all if not needed
+ * Don't include too much details about test cases
+ - Tests should be clear enough to
+ - Dublicate information is waste and maintenance problem
+ * Can contain links to more information
+
+== Test case documentation ==
+
+ * Test cases normally don't need documentation
+ - Suite's name and documentation and test's name should give enough
+ background information
+ - Test cases' structure should be clear enough without documentation
+ or other comments
+ * Sometimes useful so don't be afraid to use it
+
+== User keyword documentation ==
+
+ * Normally not needed
+ - Good name and clear structure should be enough
+ * Can be used to explain arguments and return values
+ * Visible in resource file documentations generated with libdoc.py
+
+= Test case structure =
+
+ * Test case should be easy to understand
+ * One test case should be testing one thing
+ - 'Things' can be small (part of single feature) or large (end-to-end)
+ * Select suitable abstraction level
+ - Only include information that is relevant for the test case
+ * Two kinds of test cases
+ - Workflow tests
+ - Data-driven tests
+
+== Workflow tests ==
+
+ * Steps describe what test does
+ - Use clear keyword names and suitable abstraction level
+ - Should contain enough information to run manually
+ - Should never need documentation or commenting to explain what test
does
+ * Can have different abstraction level
+ - Tests for a detailed functionality are more precise
+ - End-to-end tests can be on very high level
+ - One test should use only one abstraction level
+ * Different styles
+ - More techical tests for lower level details and integration tests
+ - "Executable specifications" act as requiremens
+ - Everyone (including customer/product owner) should always understand
+ * No complex logic
+ - No for loops or if/else constructs
+ - Use variable assignements with care
+ - Test cases should not look like scripts
+ * Max 10 steps, preferably less
+
+== Data-driven tests ==
+
+ * One high-level keyword per test
+ - Different arguments create different tests
+ - The keyword often contains similar workflow as workflow tests
+ and is created in the same test case file
+ * Possible, and recommended, to name column headings
+ * Better to have separate tests than test multiple things in one test
+ - Easier to understand than for example looping
+ - No tricks needed to go through all test even if some fails
+ - There's not that much extra work or other overhead
+ * If lots of test needed, consider generating them based on a separate
model
+
+
+= User keywords =
+
+ * Should be easy to understand
+ - Same rules as with workflow tests
+ * Different abstraction levels
+ * Can contain some programming logic (for loops, if/else)
+ - Especially on lower level keywords
+ - Complex logic in libraries rather than in user keywords
+
+
+= Variables =
+
+== Variable naming ==
+
+ * Clear but not too long names
+ * Can use comments in variable table to document them more
+ * Use case consistently
+ - Lower case with variables only available inside certain test or
keyword
+ - Upper case with others (global, suite or test level)
+ - Both space and underscore can be used as word separator
+ * Recommended to list also variables that are set dynamically in the
+ variable table
+ - Set typically using `Set Global/Suite/Test Variable` keywords
+ - The initial value should explain where/how the real value is set
+
+== Passing and returning values ==
+
+ * Common approach is to return values from keywords, assign them to
variables
+ and then pass them as arguments to other keywords
+ - Clear and easy to follow approach
+ - Looks like programming
+ * Alternative approach is using `Set Test Variable` keyword
+ - No need to have any programming style in test case level
+ - More complex to follow, reusing keywords harder
+ - Avoid below test case level
+
+
+= Avoid sleeping =
+
+ * Sleeping is very fragile
+ * Safety margins cause too long sleeps on average
+ * Instead of sleep, use keyword that polls has certain action occurred
+ - Should have a maximum time to wait
+ - Keyword names often starts with `Wait Until ...`
+ - Possible to wrap other keywords inside built-in keyword
+ `Wait Until Keyword Succeeds`
+ * Sometimes sleeping is easiest but use with care
+ - Never use in keywords that are used often by tests or other keywords
