[Laszlo-dev] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding

[Raju Bitter]

Change 20090608-raju-Y by r...@ip-90-186-160-71.web.vodafone.de on  
2009-06-08 20:09:02 CEST
    in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
    for http://rajubit...@svn.openlaszlo.org/openlaszlo/trunk
Summary: Fix for CSS parser uses incorrect file encoding

New Features: Adds an optional @charset to the stylesheet tag, in case  
the user wants to use a CSS file in a different encoding then utf-8

Bugs Fixed: LPP-8045

Technical Reviewer: ptw
QA Reviewer: (pending)
Doc Reviewer: (pending)

Documentation:

Release Notes:

Details:
+ StyleSheetCompiler.java: Handling for @charset on stylesheet tag  
added. Added a
2nd parameter with the encoding value to the CSSHandler.parse() call.

+ CSSHandler.java:
parse() method takes the encoding from the LZX stylesheet tag as a 2nd  
parameter.
getInputSource() method does a few more things now:
  1) checks for a possible BOM on the CSS file
  2) checks if a possible BOM conflicts with the value of the  
stylesheet tag's @charset
  3) if there's a BOM, the BOM bytes are removed from the input stream

+ FileUtils.java:
Added method public static String  
detectBOMEncoding(BufferedInputStream in)
The method returns the BOM marker interpreted as one of the following  
strings:
  UTF-8
  UTF-16LE
  UTF-16BE

Tests:
+ test files in folder test/css/encoding
The following test exist:
  1) iso8859-1_with_charset_attr.lzx
     Reading an iso-8859-2 encoded CSS file with some German special  
chars
  2) utf16BE_with_BOM.lzx
     Reading an utf-16 BE CSS file with BOM marker
  3) utf16LE_with_BOM.lzx
     Reading an utf-16 LE CSS file with BOM marker
  4) utf8_with_BOM_no_charset_attr.lzx
     Reading an utf-8 CSS file with BOM and no charset attribute on  
the stylesheet tag
  5) utf8_with_BOM_conflicting_charset_attr.lzx
     Reading a CSS with @charset value of utf-16, but CSS having a  
UTF-8 BOM marker, will
     throw a compile error

Files:
A       test/css/encoding
A       test/css/encoding/iso8859-1_with_charset_attr.lzx
A       test/css/encoding/utf16BE_with_BOM.lzx
A       test/css/encoding/utf8_with_BOM_no_charset_attr.css
A       test/css/encoding/utf16LE_with_BOM.css
A       test/css/encoding/utf8_no_BOM_no_charset_attr.css
A       test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
A       test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
A       test/css/encoding/utf16LE_with_BOM.lzx
A       test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
A       test/css/encoding/iso8859-1_with_charset_attr.css
A       test/css/encoding/utf16BE_with_BOM.css
A       test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
M       WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
M       WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
M       WEB-INF/lps/server/src/org/openlaszlo/compiler/ 
StyleSheetCompiler.java

Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar