On 2015-05-04 08:57:29, Kinney, Michael D wrote: > Jordan, > > Some source control systems provide support a file type of UTF-16LE, > so the use of 'binary' should be avoided. What source control > systems require the use of 'binary'?
Svn seems to require it so the data doesn't get munged. Git seems to auto-detect UTF-16 .uni files as binary. > What diff utilities are having issues with UTF-16LE files? Can you > provide some examples? I don't think it is possible to create a .uni patch file for EDK II that actually works. Normally, for .uni files we just see something about the diff being omitted on a binary file. With git, I know there are ways to force a diff to be produced, but I don't believe it would actually work to apply a patch to a UTF-16 file. With utf-8, it seems to 'just work'. (With git and svn.) > Also, what are the pros/cons to extending the exiting .uni file > extension to support either UTF16-LE or UTF-8 encodings vs. adding a > new file extension? I don't know of any pros/cons. We could assume that a file without the UTF-16 BOM is UTF-8. It would certainly mess up my unit tests. :) -Jordan > -----Original Message----- > From: Jordan Justen [mailto:jordan.l.jus...@intel.com] > Sent: Sunday, May 03, 2015 11:15 PM > To: edk2-devel@lists.sourceforge.net > Subject: [edk2] [PATCH 0/9] Support UTF-8 (.utf8) string files > > The UTF-16 .uni files are fairly annoying to work with: > * They must be checked in as 'binary' files > * It is difficult to produce a diff of changes > * UTF-8 is more likely to be supported by text editors > > With some basic testing on Linux, it appears that UTF-8 works as > expected with UTF-8 files. Both git and subversion diff output show > unicode characters > 0xff as expected. > > Note, these changes only affect the source input for unicode string > data. The output of string processing still always produces UTF-16LE > character strings. This is required since UEFI only supports UTF-16LE > string data. > > These changes are available in git at: > https://github.com/jljusten/edk2.git utf8-v1 > > Jordan Justen (9): > BaseTools/Tests: Always add BaseTools source to import path > BaseTools/EdkLogger: Support unit tests with a SILENT log level > BaseTools/Tests: Add unit test for AutoGen.UniClassObject > BaseTools: Support UTF-8 string data in .utf8 files > BaseTools/CheckUnicodeSourceFiles: Verify .utf8 string files > BaseTools/UniClassObject: Verify string data is 16-bit > BaseTools/Tests: Verify 32-bit UTF-8 chars are rejected > BaseTools/Tests: 32-bit unicode chars are allowed in comments > OvmfPkg/PlatformDxe: Convert Platform.uni to .utf8 > > BaseTools/Conf/build_rule.template | 2 +- > BaseTools/Source/Python/AutoGen/UniClassObject.py | 21 +++- > BaseTools/Source/Python/Common/EdkLogger.py | 9 +- > BaseTools/Tests/CheckUnicodeSourceFiles.py | 128 > ++++++++++++++++++++++ > BaseTools/Tests/PythonToolsTests.py | 4 +- > BaseTools/Tests/RunTests.py | 2 - > BaseTools/Tests/TestTools.py | 9 +- > OvmfPkg/PlatformDxe/Platform.inf | 2 +- > OvmfPkg/PlatformDxe/Platform.uni | Bin 3298 -> 0 bytes > OvmfPkg/PlatformDxe/Platform.utf8 | 38 +++++++ > 10 files changed, 206 insertions(+), 9 deletions(-) > create mode 100644 BaseTools/Tests/CheckUnicodeSourceFiles.py > delete mode 100644 OvmfPkg/PlatformDxe/Platform.uni > create mode 100644 OvmfPkg/PlatformDxe/Platform.utf8 > > -- > 2.1.4 > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel
