2 new revisions:
Revision: 7cf5cf21d736
Branch: default
Author: Pekka Klärck
Date: Wed Sep 19 00:51:15 2012
Log: atest: cleaned up, and made consistent, tests for data in UTF-8
encodi...
http://code.google.com/p/robotframework/source/detail?r=7cf5cf21d736
Revision: f0228803a6cf
Branch: default
Author: Pekka Klärck
Date: Wed Sep 19 03:11:15 2012
Log: Workaround for IronPython not being able to lowercase non-ASCII
charac...
http://code.google.com/p/robotframework/source/detail?r=f0228803a6cf
==============================================================================
Revision: 7cf5cf21d736
Branch: default
Author: Pekka Klärck
Date: Wed Sep 19 00:51:15 2012
Log: atest: cleaned up, and made consistent, tests for data in UTF-8
encoding in different formats
http://code.google.com/p/robotframework/source/detail?r=7cf5cf21d736
Modified:
/atest/robot/parsing/utf8_data/utf8_in_html.txt
/atest/robot/parsing/utf8_data/utf8_in_tsv.txt
/atest/robot/parsing/utf8_data/utf8_in_txt.txt
/atest/testdata/parsing/utf8_data.html
/atest/testdata/parsing/utf8_data.tsv
/atest/testdata/parsing/utf8_data.txt
=======================================
--- /atest/robot/parsing/utf8_data/utf8_in_html.txt Mon Oct 31 23:48:58 2011
+++ /atest/robot/parsing/utf8_data/utf8_in_html.txt Wed Sep 19 00:51:15 2012
@@ -1,5 +1,5 @@
*** Settings ***
-Suite Setup Run Tests ${EMPTY} parsing${/}utf8_data.html
+Suite Setup Run Tests ${EMPTY} parsing/utf8_data.html
Force Tags pybot jybot regression
Resource atest_resource.txt
@@ -12,16 +12,16 @@
UTF-8 In Keyword Arguments
${tc} = Check Test Case UTF-8
- Should Be Equal ${tc.setup.msgs[0].message} äöå
- Should Be Equal ${tc.kws[0].msgs[0].message} §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[0].msgs[0].message} äöå
- Should Be Equal ${tc.kws[2].kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[2].msgs[0].message} §½€
+ Check Log Message ${tc.setup.msgs[0]} äöå
+ Check Log Message ${tc.kws[0].msgs[0]} §½€
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[0].msgs[0]} äöå
+ Check Log Message ${tc.kws[2].kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[2].msgs[0]} §½€
UTF-8 In Test Case And UK Names
${tc} = Check Test Case UTF-8 Name Äöå §½€"
Should Be Equal ${tc.kws[0].name} \${ret} = Äöå §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].msgs[0].message} value
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[3].msgs[0]} value
=======================================
--- /atest/robot/parsing/utf8_data/utf8_in_tsv.txt Mon Oct 31 23:48:58 2011
+++ /atest/robot/parsing/utf8_data/utf8_in_tsv.txt Wed Sep 19 00:51:15 2012
@@ -1,5 +1,5 @@
*** Settings ***
-Suite Setup Run Tests ${EMPTY} parsing${/}utf8_data.tsv
+Suite Setup Run Tests ${EMPTY} parsing/utf8_data.tsv
Force Tags pybot jybot regression
Resource atest_resource.txt
@@ -12,16 +12,16 @@
UTF-8 In Keyword Arguments
${tc} = Check Test Case UTF-8
- Should Be Equal ${tc.setup.msgs[0].message} äöå
- Should Be Equal ${tc.kws[0].msgs[0].message} §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[0].msgs[0].message} äöå
- Should Be Equal ${tc.kws[2].kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[2].msgs[0].message} §½€
+ Check Log Message ${tc.setup.msgs[0]} äöå
+ Check Log Message ${tc.kws[0].msgs[0]} §½€
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[0].msgs[0]} äöå
+ Check Log Message ${tc.kws[2].kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[2].msgs[0]} §½€
UTF-8 In Test Case And UK Names
${tc} = Check Test Case UTF-8 Name Äöå §½€"
Should Be Equal ${tc.kws[0].name} \${ret} = Äöå §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].msgs[0].message} value
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[3].msgs[0]} value
=======================================
--- /atest/robot/parsing/utf8_data/utf8_in_txt.txt Mon Oct 31 23:48:58 2011
+++ /atest/robot/parsing/utf8_data/utf8_in_txt.txt Wed Sep 19 00:51:15 2012
@@ -1,5 +1,5 @@
*** Settings ***
-Suite Setup Run Tests ${EMPTY} parsing${/}utf8_data.txt
+Suite Setup Run Tests ${EMPTY} parsing/utf8_data.txt
Force Tags pybot jybot regression
Resource atest_resource.txt
@@ -12,16 +12,16 @@
UTF-8 In Keyword Arguments
${tc} = Check Test Case UTF-8
- Should Be Equal ${tc.setup.msgs[0].message} äöå
- Should Be Equal ${tc.kws[0].msgs[0].message} §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[0].msgs[0].message} äöå
- Should Be Equal ${tc.kws[2].kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].kws[2].msgs[0].message} §½€
+ Check Log Message ${tc.setup.msgs[0]} äöå
+ Check Log Message ${tc.kws[0].msgs[0]} §½€
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[0].msgs[0]} äöå
+ Check Log Message ${tc.kws[2].kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[2].kws[2].msgs[0]} §½€
UTF-8 In Test Case And UK Names
${tc} = Check Test Case UTF-8 Name Äöå §½€"
Should Be Equal ${tc.kws[0].name} \${ret} = Äöå §½€
- Should Be Equal ${tc.kws[1].msgs[0].message} äöå §½€
- Should Be Equal ${tc.kws[2].msgs[0].message} value
+ Check Log Message ${tc.kws[1].msgs[0]} äöå §½€
+ Check Log Message ${tc.kws[3].msgs[0]} value
=======================================
--- /atest/testdata/parsing/utf8_data.html Sat May 31 09:57:24 2008
+++ /atest/testdata/parsing/utf8_data.html Wed Sep 19 00:51:15 2012
@@ -219,6 +219,13 @@
<td></td>
</tr>
<tr>
+<td></td>
+<td>...</td>
+<td>FAIL Virheessäkin on ääkkösiä: Äöå §½€"</td>
+<td></td>
+<td></td>
+</tr>
+<tr>
<td></td>
<td>${ret}</td>
<td>äöå §½€</td>
@@ -233,6 +240,13 @@
<td></td>
</tr>
<tr>
+<td></td>
+<td>Should Be Equal</td>
+<td>${ret}</td>
+<td>äöå §½€</td>
+<td></td>
+</tr>
+<tr>
<td></td>
<td>Log</td>
<td>${UTF NAME öäå §½€}</td>
@@ -241,8 +255,8 @@
</tr>
<tr>
<td></td>
-<td></td>
-<td></td>
+<td>Fail</td>
+<td>Virheessäkin on ääkkösiä: Äöå §½€"</td>
<td></td>
<td></td>
</tr>
=======================================
--- /atest/testdata/parsing/utf8_data.tsv Fri Aug 14 05:12:32 2009
+++ /atest/testdata/parsing/utf8_data.tsv Wed Sep 19 00:51:15 2012
@@ -22,10 +22,12 @@
[Teardown] Logging Keyword äöå
UTF-8 Name Äöå §½€" [Documentation] Quote is actually plain ASCII but
there was a bug in processing them also.
- ${ret} äöå §½€
- Log ${ret}
+ ... FAIL Virheessäkin on ääkkösiä: Äöå §½€"
+ ${ret} äöå §½€
+ Log ${ret}
+ Should Be Equal ${ret} äöå §½€
Log ${UTF NAME öäå §½€}
-
+ Fail Virheessäkin on ääkkösiä: Äöå §½€"
*Keyword* *Action* *Argument* *Argument* *Argument* *Argument* *Argument*
*Argument*
Logging Keyword [Arguments] ${value}
=======================================
--- /atest/testdata/parsing/utf8_data.txt Thu Apr 8 14:00:02 2010
+++ /atest/testdata/parsing/utf8_data.txt Wed Sep 19 00:51:15 2012
@@ -25,6 +25,7 @@
... FAIL Virheessäkin on ääkkösiä: Äöå §½€"
${ret} = äöå §½€
Log ${ret}
+ Should Be Equal ${ret} äöå §½€
Log ${UTF NAME öäå §½€}
Fail Virheessäkin on ääkkösiä: Äöå §½€"
==============================================================================
Revision: f0228803a6cf
Branch: default
Author: Pekka Klärck
Date: Wed Sep 19 03:11:15 2012
Log: Workaround for IronPython not being able to lowercase non-ASCII
characters.
http://ironpython.codeplex.com/workitem/33133
Update issue 1237
Status: Done
Fixed with unit tests.
http://code.google.com/p/robotframework/source/detail?r=f0228803a6cf
Modified:
/src/robot/utils/__init__.py
/src/robot/utils/normalizing.py
/utest/utils/test_normalizing.py
=======================================
--- /src/robot/utils/__init__.py Wed Jun 6 08:43:20 2012
+++ /src/robot/utils/__init__.py Wed Sep 19 03:11:15 2012
@@ -33,7 +33,7 @@
from .importer import Importer
from .match import eq, matches, matches_any, Matcher, MultiMatcher
from .misc import plural_or_not, printable_name, seq2str, seq2str2, getdoc
-from .normalizing import normalize, normalize_tags, NormalizedDict
+from .normalizing import lower, normalize, normalize_tags, NormalizedDict
from .robotenv import get_env_var, set_env_var, del_env_var, get_env_vars
from .robotpath import normpath, abspath, get_link_path
from .robottime import (get_timestamp, get_start_timestamp, format_time,
=======================================
--- /src/robot/utils/normalizing.py Wed Jun 6 07:39:42 2012
+++ /src/robot/utils/normalizing.py Wed Sep 19 03:11:15 2012
@@ -13,6 +13,7 @@
# limitations under the License.
import re
+import sys
from UserDict import UserDict
@@ -28,14 +29,35 @@
if spaceless:
string = _WHITESPACE_REGEXP.sub('', string)
if caseless:
- string = string.lower()
- ignore = [i.lower() for i in ignore]
+ string = lower(string)
+ ignore = [lower(i) for i in ignore]
for ign in ignore:
if ign in string: # performance optimization
string = string.replace(ign, '')
return string
+# IronPython fails to lowercase non-ASCII characters:
+# http://ironpython.codeplex.com/workitem/33133
+if sys.platform != 'cli':
+ def lower(string):
+ return string.lower()
+
+else:
+ def lower(string):
+ if string.islower():
+ return string
+ if not _has_non_ascii_chars(string):
+ return string.lower()
+ return ''.join(c if not c.isupper() else c.swapcase() for c in
string)
+
+ def _has_non_ascii_chars(string):
+ for c in string:
+ if c >= u'\x80':
+ return True
+ return False
+
+
# TODO: Move normalizing tags to robot.model.Tags in 2.8.
# Move can be done after all tag related code uses Tags.
@@ -56,7 +78,7 @@
class NormalizedDict(UserDict):
"""Custom dictionary implementation automatically normalizing keys."""
- def __init__(self, initial=None, ignore=[], caseless=True,
spaceless=True):
+ def __init__(self, initial=None, ignore=(), caseless=True,
spaceless=True):
"""Initializes with possible initial value and normalizing spec.
Initial values can be either a dictionary or an iterable of
name/value
=======================================
--- /utest/utils/test_normalizing.py Wed Dec 7 07:28:15 2011
+++ /utest/utils/test_normalizing.py Wed Sep 19 03:11:15 2012
@@ -23,6 +23,10 @@
assert_equals(normalize('Fo o BaR', caseless=False), 'FooBaR')
assert_equals(normalize('Fo O B AR', caseless=True), 'foobar')
+ def test_normalize_with_caseless_non_ascii(self):
+ assert_equals(normalize(u'\xc4iti', caseless=False), u'\xc4iti')
+ assert_equals(normalize(u'\xc4iti', caseless=True), u'\xe4iti')
+
def test_normalize_with_spaceless(self):
assert_equals(normalize('Fo o BaR', spaceless=False), 'fo o bar')
assert_equals(normalize('Fo O B AR', spaceless=True), 'foobar')
@@ -116,6 +120,15 @@
assert_raises(KeyError, nd2.__getitem__, key)
assert_true(key not in nd2)
+ def test_caseless_with_non_ascii(self):
+ nd1 = NormalizedDict({u'\xe4': 1})
+ assert_equals(nd1[u'\xe4'], 1)
+ assert_equals(nd1[u'\xc4'], 1)
+ assert_true(u'\xc4' in nd1)
+ nd2 = NormalizedDict({u'\xe4': 1}, caseless=False)
+ assert_equals(nd2[u'\xe4'], 1)
+ assert_true(u'\xc4' not in nd2)
+
def test_has_key_and_contains(self):
nd = NormalizedDict({'Foo': 'bar'})
assert_true(nd.has_key('Foo') and nd.has_key(' f O o '))
