See <http://robot.radiaatto.ri.fi/hudson/job/RobotFrameworkJython/1475/changes>
Changes: [Pekka Klärck] CommandLineMonitor: Write kw markers using green '.' when kw passes and with red 'F' when it fails. Update issue 1066 This feature is getting ready. Things that have been resolved since the last comment to this issue: - Markers are not used when stdout is redirected. - Different markers used depending kw status, either green `.` or red `F` - Warnings during execution do not mess up markers (they are actually handled better than earlier) Remaining tasks: - Testing on Windows with all interpreters - Code review and possible refactoring - Investigate could this be tested somehow (can be hard automatically as the feature is turned off when outputs are captured) - Consider writing something to UG [Pekka Klärck] CommandLineMonitor: write kw markers correctly when there are warnings during execution -- as a bonus warnings are now aligned better than they used to [Pekka Klärck] CommandLineMonitor: Simpler status highlighting. Old system looked generic but wasn't. [Pekka Klärck] monitor: cleanup [Pekka Klärck] testdoc: use just the suite name as the default title, some usage doc cleanup [Pekka Klärck] testdoc: 1) tests for not escaping when there's no need, 2) renamed escaping/formatting test data file [Pekka Klärck] testdoc: fixed html formatting docs and metadata [Pekka Klärck] testdoc: fixed suite's metadata -- template expected it to be a list (which is correct as it keeps order correct) [Pekka Klärck] testdoc: 1) test data for testing formatting (actual tests and fixes will follow), 2) adapted old tests accordingly after adding new test case file under misc [Pekka Klärck] atests: fixed tests after changes to test data [Pekka Klärck] CommandLineMonitor: 1) disable keyword markers when not stdout.isatty(), 2) only write keyword markers when running tests [Pekka Klärck] misc data: changed format and added suite setup/teardown -- we'll see do any tests break because of this [Pekka Klärck] CommandLineMonitor: Notifications on console when top-level keywords end Update issue 1066 Status: Started Intial implementation of this feature is now ready. It works so that when tests are executing, there are dots added after the test name into the place where the test status is written in the end. A new dot is written every time when a top-level keyword ends. If there are more dots than there is room reserved for status, old dots are cleared. Dots are also always creared when the status is written. Tasks to do: - Test how dots work when output is redirected and possibly disable them in that case - Testing on Windows - Testing with IronPython - Investigate could dots be colored based on status - Review [Pekka Klärck] fixed utest after changing command line monitor internals [Pekka Klärck] CommandLineMonitor: separated writing to CommandLineWriter and some other cleanup [Pekka Klärck] report: made all status colors !important to allow using them in links. no need for separate summary_pass/fail anymore. [Pekka Klärck] webcontent: merged class=header and id=header_div into id=header [Pekka Klärck] regen [Pekka Klärck] regen [Pekka Klärck] testdoc: Documentation update Update issue 1064 Status: Started Implementation and now also docs ought to be done. Remaining tasks are: - Final doc review (both --help and UG) after my latest changes. - Few acceptance tests. [Pekka Klärck] removed old testdoc [Pekka Klärck] testdoc: Cleaned up setting title and generated times [Pekka Klärck] testdoc.html: indentation [Pekka Klärck] common.css: header tweaking ------------------------------------------ [...truncated 7003 lines...] ------------------------------------------------------------------------------ Test Status Should Not Exist Outside Teardown Test Status Should Not Exist Outside Teardown | PASS | ------------------------------------------------------------------------------ Test Message Should Not Exist Outside Teardown Test Message Should Not Exist Outside Teardown | PASS | ------------------------------------------------------------------------------ Test Status When Test Fails Test Status When Test Fails | PASS | ------------------------------------------------------------------------------ Test Status When Setup Fails Test Status When Setup Fails | PASS | ------------------------------------------------------------------------------ Previous Test Variables Should Have Correct Values When That Test ... Previous Test Variables Should Have Correct Values When That Test ... | PASS | ------------------------------------------------------------------------------ Previous Test Variables Should Have Default Values From Previous S... Previous Test Variables Should Have Default Values From Previous S... | PASS | ------------------------------------------------------------------------------ Suite And Prev Test Variables Work Correctly In Setup Suite And Prev Test Variables Work Correctly In Setup | PASS | ------------------------------------------------------------------------------ Suite And Prev Test Variables Work Correctly In Teardown Suite And Prev Test Variables Work Correctly In Teardown | PASS | ------------------------------------------------------------------------------ Suite Variables In Test Suite Variables In Test | PASS | ------------------------------------------------------------------------------ Robot.Variables.Test And Suite Status Variables | PASS | 13 critical tests, 13 passed, 0 failed 13 tests total, 13 passed, 0 failed ============================================================================== Robot.Variables.Variable File Implemented As Class ============================================================================== Python Class Python Class | PASS | ------------------------------------------------------------------------------ Methods in Python Class Do Not Create Variables Methods in Python Class Do Not Create Variables | PASS | ------------------------------------------------------------------------------ Dynamic Python Class Dynamic Python Class | PASS | ------------------------------------------------------------------------------ Java Class Java Class | PASS | ------------------------------------------------------------------------------ Methods in Java Class Do Not Create Variables Methods in Java Class Do Not Create Variables | PASS | ------------------------------------------------------------------------------ Dynamic Java Class Dynamic Java Class | PASS | ------------------------------------------------------------------------------ Instantiating Fails Instantiating Fails | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variable File Implemented As Class | PASS | 7 critical tests, 7 passed, 0 failed 7 tests total, 7 passed, 0 failed ============================================================================== Robot.Variables.Variable Priorities :: Some of these tests are testing same... ============================================================================== Individual CLI Variables Override All Other Variables Individual CLI Variables Override All Other Variables | PASS | ------------------------------------------------------------------------------ Variable Files From CLI Override All Variables In Test Data Variable Files From CLI Override All Variables In Test Data | PASS | ------------------------------------------------------------------------------ When Multiple Variable Files Are Given From CLI The First One Has ... When Multiple Variable Files Are Given From CLI The First One Has ... | PASS | ------------------------------------------------------------------------------ Variable Tables In test Case Files Override Variables From Resourc... Variable Tables In test Case Files Override Variables From Resourc... | PASS | ------------------------------------------------------------------------------ Variable Tables In Resource Files Override Variables From Resource... Variable Tables In Resource Files Override Variables From Resource... | PASS | ------------------------------------------------------------------------------ When Multiple Resource Or Variable Files Are Imported The First On... When Multiple Resource Or Variable Files Are Imported The First On... | PASS | ------------------------------------------------------------------------------ Variables With Different Priorities Are Seen Also In User Keywords Variables With Different Priorities Are Seen Also In User Keywords | PASS | ------------------------------------------------------------------------------ Variables Set During Test Execution Override All Variables In Thei... Variables Set During Test Execution Override All Variables In Thei... | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variable Priorities :: Some of these tests are tes... | PASS | 8 critical tests, 8 passed, 0 failed 8 tests total, 8 passed, 0 failed ============================================================================== Robot.Variables.Variable Scopes :: These tests are somewhat related to test... ============================================================================== Variables Set In Test Case Are Seen By User Keywords Variables Set In Test Case Are Seen By User Keywords | PASS | ------------------------------------------------------------------------------ Variables Set In One Test Are Not Visible In Another Variables Set In One Test Are Not Visible In Another | PASS | ------------------------------------------------------------------------------ Variables Set In User Keyword Are Seen Only By Lower Level User Ke... Variables Set In User Keyword Are Seen Only By Lower Level User Ke... | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variable Scopes :: These tests are somewhat relate... | PASS | 3 critical tests, 3 passed, 0 failed 3 tests total, 3 passed, 0 failed ============================================================================== Robot.Variables.Variable Table ============================================================================== Scalar String Scalar String | PASS | ------------------------------------------------------------------------------ Scalar Non-Strings Scalar Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar String With Escapes Scalar String With Escapes | PASS | ------------------------------------------------------------------------------ Empty Scalar String Empty Scalar String | PASS | ------------------------------------------------------------------------------ Scalar List Scalar List | PASS | ------------------------------------------------------------------------------ Scalar List With Non-Strings Scalar List With Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar List With Escapes Scalar List With Escapes | PASS | ------------------------------------------------------------------------------ List With One Item List With One Item | PASS | ------------------------------------------------------------------------------ List With Multiple Items List With Multiple Items | PASS | ------------------------------------------------------------------------------ List With Escapes List With Escapes | PASS | ------------------------------------------------------------------------------ List Created From List With Escapes List Created From List With Escapes | PASS | ------------------------------------------------------------------------------ List With No Items List With No Items | PASS | ------------------------------------------------------------------------------ Scalar Variable Containing List Is Different Than List Variable Scalar Variable Containing List Is Different Than List Variable | PASS | ------------------------------------------------------------------------------ Variable Names Are Case Insensitive Variable Names Are Case Insensitive | PASS | ------------------------------------------------------------------------------ Variable Names Are Space Insensitive Variable Names Are Space Insensitive | PASS | ------------------------------------------------------------------------------ Variable Names Are Underscore Insensitive Variable Names Are Underscore Insensitive | PASS | ------------------------------------------------------------------------------ Assign Mark With Scalar Variable Assign Mark With Scalar Variable | PASS | ------------------------------------------------------------------------------ Assign Mark With List Variable Assign Mark With List Variable | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variable Table | PASS | 18 critical tests, 18 passed, 0 failed 18 tests total, 18 passed, 0 failed ============================================================================== Robot.Variables.Variables From Resource Files ============================================================================== Scalar String Scalar String | PASS | ------------------------------------------------------------------------------ Scalar Non-Strings Scalar Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar String With Escapes Scalar String With Escapes | PASS | ------------------------------------------------------------------------------ Empty Scalar String Empty Scalar String | PASS | ------------------------------------------------------------------------------ Scalar List Scalar List | PASS | ------------------------------------------------------------------------------ Scalar List With Non-Strings Scalar List With Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar List With Escapes Scalar List With Escapes | PASS | ------------------------------------------------------------------------------ List with One Item List with One Item | PASS | ------------------------------------------------------------------------------ List With Multiple Items List With Multiple Items | PASS | ------------------------------------------------------------------------------ List With Escapes List With Escapes | PASS | ------------------------------------------------------------------------------ List Created From List With Escapes List Created From List With Escapes | PASS | ------------------------------------------------------------------------------ List With No Items List With No Items | PASS | ------------------------------------------------------------------------------ Scalar Variable Containing List Is Different Than List Variable Scalar Variable Containing List Is Different Than List Variable | PASS | ------------------------------------------------------------------------------ Variable Names Are Case Insensitive Variable Names Are Case Insensitive | PASS | ------------------------------------------------------------------------------ Variable Names Are Space Insensitive Variable Names Are Space Insensitive | PASS | ------------------------------------------------------------------------------ Variable Names Are Underscore Insensitive Variable Names Are Underscore Insensitive | PASS | ------------------------------------------------------------------------------ Assign Mark With Scalar variable Assign Mark With Scalar variable | PASS | ------------------------------------------------------------------------------ Assign Mark With List variable Assign Mark With List variable | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variables From Resource Files | PASS | 18 critical tests, 18 passed, 0 failed 18 tests total, 18 passed, 0 failed ============================================================================== Robot.Variables.Variables From Variable Files ============================================================================== Scalar String Scalar String | PASS | ------------------------------------------------------------------------------ Scalar Non-Strings Scalar Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar String With Escapes Scalar String With Escapes | PASS | ------------------------------------------------------------------------------ Empty Scalar String Empty Scalar String | PASS | ------------------------------------------------------------------------------ Scalar List Scalar List | PASS | ------------------------------------------------------------------------------ Scalar List With Non-Strings Scalar List With Non-Strings | PASS | ------------------------------------------------------------------------------ Scalar List With Escapes Scalar List With Escapes | PASS | ------------------------------------------------------------------------------ Scalar Object Scalar Object | PASS | ------------------------------------------------------------------------------ List with One Item List with One Item | PASS | ------------------------------------------------------------------------------ List With Multiple Items List With Multiple Items | PASS | ------------------------------------------------------------------------------ List With Escapes List With Escapes | PASS | ------------------------------------------------------------------------------ List With No Items List With No Items | PASS | ------------------------------------------------------------------------------ List With Objects List With Objects | PASS | ------------------------------------------------------------------------------ Scalar Variable Containing List Is Different Than List Variable Scalar Variable Containing List Is Different Than List Variable | PASS | ------------------------------------------------------------------------------ Variable Names Are Case Insensitive Variable Names Are Case Insensitive | PASS | ------------------------------------------------------------------------------ Variable Names Are Underscore Insensitive Variable Names Are Underscore Insensitive | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variables From Variable Files | PASS | 16 critical tests, 16 passed, 0 failed 16 tests total, 16 passed, 0 failed ============================================================================== Robot.Variables.Variables In Import Settings ============================================================================== Variable Defined In Test Case File Is Used To Import Resources Variable Defined In Test Case File Is Used To Import Resources | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variables In Import Settings | PASS | 1 critical test, 1 passed, 0 failed 1 test total, 1 passed, 0 failed ============================================================================== Robot.Variables.Variables Inside Variables ============================================================================== Variable Inside Variable In Variable Table Variable Inside Variable In Variable Table | PASS | ------------------------------------------------------------------------------ Variable Inside Variable In Test Case Variable Inside Variable In Test Case | PASS | ------------------------------------------------------------------------------ Variable Inside Variable In User Keyword Variable Inside Variable In User Keyword | PASS | ------------------------------------------------------------------------------ Variable Inside List Variable Variable Inside List Variable | PASS | ------------------------------------------------------------------------------ Variable Inside List Variable Index Variable Inside List Variable Index | PASS | ------------------------------------------------------------------------------ Variable Inside Variable And Extended Variable Syntax Variable Inside Variable And Extended Variable Syntax | PASS | ------------------------------------------------------------------------------ Non-Existing Variable Inside Variable Non-Existing Variable Inside Variable | PASS | ------------------------------------------------------------------------------ Robot.Variables.Variables Inside Variables | PASS | 7 critical tests, 7 passed, 0 failed 7 tests total, 7 passed, 0 failed ============================================================================== Robot.Variables | PASS | 182 critical tests, 182 passed, 0 failed 182 tests total, 182 passed, 0 failed ============================================================================== Robot :: Robot Framework acceptance tests | FAIL | 2510 critical tests, 2506 passed, 4 failed 2537 tests total, 2506 passed, 31 failed ============================================================================== Output: <http://robot.radiaatto.ri.fi/hudson/job/RobotFrameworkJython/ws/atest/results/output.xml> Log: <http://robot.radiaatto.ri.fi/hudson/job/RobotFrameworkJython/ws/atest/results/log.html> Report: <http://robot.radiaatto.ri.fi/hudson/job/RobotFrameworkJython/ws/atest/results/report.html> Build step 'Execute shell' marked build as failure Robot results publisher started... -Parsing output xml: Done! -Copying log files to build dir: Done! -Assigning results to build: Done! -Checking thresholds: Done! Done publishing Robot results.
