Hi Dan,
It is very important that CATS-0.2 be design with internationalization
support in mind.
The most common issues / bugs that arise related to i18n are subtle and
a good testing
framework is essential.
Could you detail what work / thought has gone in to CATS-0.2 regarding
i18n and what is left to be done.
Specifically in CATS-0.1 I went through and wrapped all displayable
strings in uw() methods from the
i18n.tests package. The uw method inserted a semi-random unicode
character at the beginning of the
displayable string and a unicode character at the end of the displayable
string.
Adding this uw() wrap was key to finding many subtle bugs including
incorrectly logging
unicode characters and not encoding unicode to bytes when using the
Python file system API's.
It even uncovered issues with the CATS-0.1 framework itself including a
few places where items were
converted to strings i.e. str(item) which is a no no!
Using unicode strings is only a small aspect of an i18n testing framework.
In CATS I would like the framework to provide support for running in
different locales and of course timezones.
For each locale the expected output for many operations will change. For
example a date / time string will
be different, the first day or the week in a calendar can be different,
The UI menu labels and auto generated
collection names will be different.
I am happy to sync up with you guys but I do want to make sure at this
stage we are writing frameworks that
meet the needs of Chandler.
Almost all of the Internationalization work will be completed in the
Alpha 4 time frame.
The testing framework is the key to preventing regression bugs in i18n
as we drive towards Chandler 1.0.
Thanks,
Brian
Dan Steinicke wrote:
We are getting close to being able to switch over to the new test
framework (CATS-0.2). As it is currently implemented the output from
the tests will be quite different from the current test output.
Please take a moment to look over the test output samples below and
let us know what you think so we can address peoples concerns before
the new framework goes live, rather than after.
Two sample outputs are shown below, one where a test fails without a
traceback, another of a failure with a traceback.
For more info on the new framework see :
general docs
http://wiki.osafoundation.org/bin/view/Projects/ChandlerAutomatedTestSystemZeroPointTwo
writing new tests
http://wiki.osafoundation.org/bin/view/Projects/WritingChandlerAutomatedTestsWithCATSZeroPointTwo
Thanks
Dan
############# Here is sample output from do_tests where one tests
fails without a traceback:
Test Report;
*Suite ""ChandlerTestSuite"" Failed :: Total Time ""0:04:55.434000""
:: Comment ""None""
**Test ""TestSwitchTimezone"" Failed :: Total Time ""0:00:00.931000""
:: Comment ""None
None""
***Action ""CheckBlockVisibility"" Failed :: Total Time ""0:00:00"" ::
Comment ""(On EditTimeZone Visibility) || detail view = Fa
lse ; expected value = True""
****Report ""(On EditTimeZone Visibility) || detail view = False ;
expected value = True"" Failed :: Comment ""None""
***Action ""CheckEditableBlock"" Failed :: Total Time ""0:00:00"" ::
Comment ""(On EditTimeZone Checking) || detail view value =
Floating ; expected value = US/Pacific""
****Report ""(On EditTimeZone Checking) || detail view value =
Floating ; expected value = US/Pacific"" Failed :: Comment ""None"
"
***Action ""CheckBlockVisibility"" Failed :: Total Time ""0:00:00"" ::
Comment ""(On EditTimeZone Visibility) || detail view = Fa
lse ; expected value = True""
****Report ""(On EditTimeZone Visibility) || detail view = False ;
expected value = True"" Failed :: Comment ""None""
***Action ""CheckEditableBlock"" Failed :: Total Time ""0:00:00"" ::
Comment ""(On EditTimeZone Checking) || detail view value =
Floating ; expected value = US/Pacific""
****Report ""(On EditTimeZone Checking) || detail view value =
Floating ; expected value = US/Pacific"" Failed :: Comment ""None"
"
$Suites run=1, pass=0, fail=1 :: Tests run=26, pass=25, fail=1 ::
Actions run=367, pass=363, fail=4 :: Reports run=556, pass=552,
fail=4
#TINDERBOX# Testname = ChandlerTestSuite
#TINDERBOX# Time elapsed = 0:04:55.434000 (seconds)
#TINDERBOX# Status = FAILED
- + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
The following tests failed
(debug)C:\cygwin\home\Dan\chandler\tools\cats\Functional\FunctionalTestSuite.py
(release)C:\cygwin\home\Dan\chandler\tools\cats\Functional\FunctionalTestSuite.py
########## here is some sample output of a test failing with a traceback
Test Report;
*Suite ""ChandlerTestSuite"" Failed :: Total Time ""0:00:06.139000""
:: Comment ""None""
**Test ""TestCauseTrace"" Failed :: Total Time ""0:00:00.040000"" ::
Comment ""None
Test Failure due to traceback
Traceback (most recent call last):
File "C:\cygwin\home\Dan\chandler\tools\cats\framework\runTests.py",
line 51, in run_tests
test.runTest()
File
"C:\cygwin\home\Dan\chandler\tools\cats\framework\ChandlerTestCase.py",
line 68, in runTest
self.startTest()
File
"C:\cygwin\home\Dan\chandler\tools\cats\Functional\TestCauseTrace.py",
line 24, in startTest
1/0
ZeroDivisionError: integer division or modulo by zero
""
***Action ""Divide by zero "" Failed :: Total Time ""0:00:00.040000""
:: Comment ""None""
****Report ""Action Failure due to traceback"" Failed :: Comment ""None""
$Suites run=1, pass=0, fail=1 :: Tests run=3, pass=2, fail=1 ::
Actions run=5, pass=4, fail=1 :: Reports run=28, pass=27, fail=1
#TINDERBOX# Testname = ChandlerTestSuite
#TINDERBOX# Time elapsed = 0:00:06.139000 (seconds)
#TINDERBOX# Status = FAILED
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
--
Brian Kirsch
Internationalization Architect/ Mail Service Engineer
Open Source Applications Foundation
543 Howard Street 5th Floor
San Francisco, CA 94105
http://www.osafoundation.org
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
