Revision: 17698
http://sourceforge.net/p/edk2/code/17698
Author: jljusten
Date: 2015-06-23 23:34:38 +0000 (Tue, 23 Jun 2015)
Log Message:
-----------
BaseTools/Tests: Verify unsupported UTF-8 data is rejected
Surrogate pair characters can be encoded in UTF-8 files, but they are
not valid UCS-2 characters.
For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\ud801', 'utf-8')
'\xed\xa0\x81'
But, the range of 0xd800 - 0xdfff should be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.
We test that this case is rejected for UTF-8 with and without the
UTF-8 BOM.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <[email protected]>
Reviewed-by: Laszlo Ersek <[email protected]>
Reviewed-by: Michael D Kinney <[email protected]>
Reviewed-by: Yingke Liu <[email protected]>
Modified Paths:
--------------
trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py
Modified: trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py
===================================================================
--- trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py 2015-06-23
23:34:33 UTC (rev 17697)
+++ trunk/edk2/BaseTools/Tests/CheckUnicodeSourceFiles.py 2015-06-23
23:34:38 UTC (rev 17698)
@@ -139,6 +139,30 @@
self.CheckFile('utf_8', shouldPass=False, string=data)
+ def testSurrogatePairUnicodeCharInUtf8File(self):
+ #
+ # Surrogate Pair code points are used in UTF-16 files to
+ # encode the Supplementary Plane characters. In UTF-8, it is
+ # trivial to encode these code points, but they are not valid
+ # code points for characters, since they are reserved for the
+ # UTF-16 Surrogate Pairs.
+ #
+ # This test makes sure that BaseTools rejects these characters
+ # if seen in a .uni file.
+ #
+ data = '\xed\xa0\x81'
+
+ self.CheckFile(encoding=None, shouldPass=False, string=data)
+
+ def testSurrogatePairUnicodeCharInUtf8FileWithBom(self):
+ #
+ # Same test as testSurrogatePairUnicodeCharInUtf8File, but add
+ # the UTF-8 BOM
+ #
+ data = codecs.BOM_UTF8 + '\xed\xa0\x81'
+
+ self.CheckFile(encoding=None, shouldPass=False, string=data)
+
TheTestSuite = TestTools.MakeTheTestSuite(locals())
if __name__ == '__main__':
------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
edk2-commits mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-commits
