Revision: 17695
http://sourceforge.net/p/edk2/code/17695
Author: jljusten
Date: 2015-06-23 23:34:22 +0000 (Tue, 23 Jun 2015)
Log Message:
-----------
BaseTools/Tests: Verify unsupported UTF-16 are rejected
Supplementary Plane characters can exist in UTF-16 files,
but they are not valid UCS-2 characters.
For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\U00010300', 'utf-16')
'\xff\xfe\x00\xd8\x00\xdf'
Therefore the UCS-4 0x00010300 character is encoded as two
16-bit numbers (0xd800 0xdf00) in a little endian UTF-16
file.
For more information, see:
http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF
This test checks to make sure that BaseTools will reject these
characters in UTF-16 files.
The range of 0xd800 - 0xdfff should also be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.
This test was fixed by the previous commit:
"BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni files"
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <[email protected]>
Reviewed-by: Laszlo Ersek <[email protected]>
Reviewed-by: Michael D Kinney <[email protected]>
Reviewed-by: Yingke Liu <[email protected]>
Modified Paths:
--------------
trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py
Modified: trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py
===================================================================
--- trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py 2015-06-23
23:34:19 UTC (rev 17694)
+++ trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py 2015-06-23
23:34:22 UTC (rev 17695)
@@ -38,7 +38,10 @@
def EncodeToFile(self, encoding, string=None):
if string is None:
string = self.SampleData
- data = codecs.encode(string, encoding)
+ if encoding is not None:
+ data = codecs.encode(string, encoding)
+ else:
+ data = string
path = 'input.uni'
self.WriteTmpFile(path, data)
return PathClass(self.GetTmpFilePath(path))
@@ -81,6 +84,36 @@
def testUtf16InUniFile(self):
self.CheckFile('utf_16', shouldPass=True)
+ def testSupplementaryPlaneUnicodeCharInUtf16File(self):
+ #
+ # Supplementary Plane characters can exist in UTF-16 files,
+ # but they are not valid UCS-2 characters.
+ #
+ # This test makes sure that BaseTools rejects these characters
+ # if seen in a .uni file.
+ #
+ data = u'''
+ #langdef en-US "English"
+ #string STR_A #language en-US "CodePoint (\U00010300) > 0xFFFF"
+ '''
+
+ self.CheckFile('utf_16', shouldPass=False, string=data)
+
+ def testSurrogatePairUnicodeCharInUtf16File(self):
+ #
+ # Surrogate Pair code points are used in UTF-16 files to
+ # encode the Supplementary Plane characters. But, a Surrogate
+ # Pair code point which is not followed by another Surrogate
+ # Pair code point might be interpreted as a single code point
+ # with the Surrogate Pair code point.
+ #
+ # This test makes sure that BaseTools rejects these characters
+ # if seen in a .uni file.
+ #
+ data = codecs.BOM_UTF16_LE + '//\x01\xd8 '
+
+ self.CheckFile(encoding=None, shouldPass=False, string=data)
+
TheTestSuite = TestTools.MakeTheTestSuite(locals())
if __name__ == '__main__':
------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
edk2-commits mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-commits
