[cc-ing Wanli]
Wanli,
Maybe you also want to review (or have someone on your team review) as
this issue affects you?
---
Raju,
This looks good.
I think you might have trouble trying checking in your utf-16 examples
to svn. I'm not sure svn will do the right thing. It looks like it
is treating them as binary files. Maybe that will work. I don't know.
It looks to me like you may also have some tab characters in your
sources. You need to expand tabs to check in, otherwise the pre-
commit filter will reject your checkin.
I approve this change. You might want to check in your fix separately
from your tests, just in case the UTF-16 files cause svn issues.
On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
Added a "+" in debug output string concatenation.
Change 20090608-raju-Y by [email protected] on
2009-06-08 20:09:02 CEST
in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
for http://[email protected]/openlaszlo/trunk
Summary: Fix for CSS parser uses incorrect file encoding
New Features: Adds an optional @charset to the stylesheet tag, in
case the user wants to use a CSS file in a different encoding then
utf-8
Bugs Fixed: LPP-8045
Technical Reviewer: ptw
QA Reviewer: (pending)
Doc Reviewer: (pending)
Documentation:
Release Notes:
Details:
+ StyleSheetCompiler.java: Handling for @charset on stylesheet tag
added. Added a
2nd parameter with the encoding value to the CSSHandler.parse() call.
+ CSSHandler.java:
parse() method takes the encoding from the LZX stylesheet tag as a
2nd parameter.
getInputSource() method does a few more things now:
1) checks for a possible BOM on the CSS file
2) checks if a possible BOM conflicts with the value of the
stylesheet tag's @charset
3) if there's a BOM, the BOM bytes are removed from the input stream
+ FileUtils.java:
Added method public static String
detectBOMEncoding(BufferedInputStream in)
The method returns the BOM marker interpreted as one of the
following strings:
UTF-8
UTF-16LE
UTF-16BE
Tests:
+ test files in folder test/css/encoding
The following test exist:
1) iso8859-1_with_charset_attr.lzx
Reading an iso-8859-2 encoded CSS file with some German special
chars
2) utf16BE_with_BOM.lzx
Reading an utf-16 BE CSS file with BOM marker
3) utf16LE_with_BOM.lzx
Reading an utf-16 LE CSS file with BOM marker
4) utf8_with_BOM_no_charset_attr.lzx
Reading an utf-8 CSS file with BOM and no charset attribute on
the stylesheet tag
5) utf8_with_BOM_conflicting_charset_attr.lzx
Reading a CSS with @charset value of utf-16, but CSS having a
UTF-8 BOM marker, will
throw a compile error
Files:
A test/css/encoding
A test/css/encoding/iso8859-1_with_charset_attr.lzx
A test/css/encoding/utf16BE_with_BOM.lzx
A test/css/encoding/utf8_with_BOM_no_charset_attr.css
A test/css/encoding/utf16LE_with_BOM.css
A test/css/encoding/utf8_no_BOM_no_charset_attr.css
A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
A test/css/encoding/utf16LE_with_BOM.lzx
A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
A test/css/encoding/iso8859-1_with_charset_attr.css
A test/css/encoding/utf16BE_with_BOM.css
A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
M WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
M WEB-INF/lps/server/src/org/openlaszlo/compiler/
StyleSheetCompiler.java
Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
