Tucker, Wanli,
I've done some testing with OL 4.0.15 and Webtop running 1.5.2 running
on OL 4.0.15. While the fix works for OL trunk, it doesn't work for
4.0.15. Here's the debug output for test file "test/css/encoding/
utf8_with_BOM_no_charset_attr.lzx". The UTF-8 encoded data is read
correctly.
http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler - StyleSheetCompiler.compile called!
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler - @charset=utf-8 found on stylesheet tag
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler - reading in stylesheet from
src="utf8_with_BOM_no_charset_attr.css"
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/src/
svn/openlaszlo/4.0.15/test/css/encoding/
utf8_with_BOM_no_charset_attr.css
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
addFile Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/
encoding/utf8_with_BOM_no_charset_attr.css
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
creating CSSHandler
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
Trying to parse CSS with charset setting of utf-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Read the following bytes: text[name='german'] {
backgroundColor: #ffcc00;
buttonText: "öäüÖÄܧ";
}
text[nam
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Testing for UTF-8 BOM!
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Found BOM on file, encoding is UTF-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
Opening CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
css/encoding/utf8_with_BOM_no_charset_attr.css using encoding UTF-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
Skip first 3 bytes containing UTF-8 BOM
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - compiling CSSHandler using new unique
names
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Conditional selector: [name="german"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Cond string: { attrname: "name",
attrvalue: "german", simpleselector: "text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Conditional selector: [name="korean"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Cond string: { attrname: "name",
attrvalue: "korean", simpleselector: "text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Conditional selector: [name="chinese"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - Cond string: { attrname: "name",
attrvalue: "chinese", simpleselector: "text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler - whole stylesheet as css $lzc
$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
"german", simpleselector: "text"}, {backgroundColor: 0xFFCC00,
buttonText: "???????"}));
$lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
"korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
buttonText: "? ?? ??? ?? ?? ?? ?? ??"}));
$lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
"chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
buttonText: "???????????????"}))
But the debug console in the browser shows the following error message:
ERROR: Unknown cohort for rule: #0 * (undefined)
ERROR: Unknown cohort for rule: #1 * (undefined)
ERROR: Unknown cohort for rule: #2 * (undefined)
WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
node «lz.text#3| #german» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
node «lz.text#3| #german» for property name backgroundColor
WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
node «lz.text#4| #korean» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
node «lz.text#4| #korean» for property name backgroundColor
WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
node «lz.text#5| #chinese» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
node «lz.text#5| #chinese» for property name backgroundColor
Any idea what this might be?
- Raju
On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
[cc-ing Wanli]
Wanli,
Maybe you also want to review (or have someone on your team review)
as this issue affects you?
---
Raju,
This looks good.
I think you might have trouble trying checking in your utf-16
examples to svn. I'm not sure svn will do the right thing. It
looks like it is treating them as binary files. Maybe that will
work. I don't know.
It looks to me like you may also have some tab characters in your
sources. You need to expand tabs to check in, otherwise the pre-
commit filter will reject your checkin.
I approve this change. You might want to check in your fix
separately from your tests, just in case the UTF-16 files cause svn
issues.
On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
Added a "+" in debug output string concatenation.
Change 20090608-raju-Y by [email protected] on
2009-06-08 20:09:02 CEST
in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
for http://[email protected]/openlaszlo/trunk
Summary: Fix for CSS parser uses incorrect file encoding
New Features: Adds an optional @charset to the stylesheet tag, in
case the user wants to use a CSS file in a different encoding then
utf-8
Bugs Fixed: LPP-8045
Technical Reviewer: ptw
QA Reviewer: (pending)
Doc Reviewer: (pending)
Documentation:
Release Notes:
Details:
+ StyleSheetCompiler.java: Handling for @charset on stylesheet tag
added. Added a
2nd parameter with the encoding value to the CSSHandler.parse() call.
+ CSSHandler.java:
parse() method takes the encoding from the LZX stylesheet tag as a
2nd parameter.
getInputSource() method does a few more things now:
1) checks for a possible BOM on the CSS file
2) checks if a possible BOM conflicts with the value of the
stylesheet tag's @charset
3) if there's a BOM, the BOM bytes are removed from the input stream
+ FileUtils.java:
Added method public static String
detectBOMEncoding(BufferedInputStream in)
The method returns the BOM marker interpreted as one of the
following strings:
UTF-8
UTF-16LE
UTF-16BE
Tests:
+ test files in folder test/css/encoding
The following test exist:
1) iso8859-1_with_charset_attr.lzx
Reading an iso-8859-2 encoded CSS file with some German special
chars
2) utf16BE_with_BOM.lzx
Reading an utf-16 BE CSS file with BOM marker
3) utf16LE_with_BOM.lzx
Reading an utf-16 LE CSS file with BOM marker
4) utf8_with_BOM_no_charset_attr.lzx
Reading an utf-8 CSS file with BOM and no charset attribute on
the stylesheet tag
5) utf8_with_BOM_conflicting_charset_attr.lzx
Reading a CSS with @charset value of utf-16, but CSS having a
UTF-8 BOM marker, will
throw a compile error
Files:
A test/css/encoding
A test/css/encoding/iso8859-1_with_charset_attr.lzx
A test/css/encoding/utf16BE_with_BOM.lzx
A test/css/encoding/utf8_with_BOM_no_charset_attr.css
A test/css/encoding/utf16LE_with_BOM.css
A test/css/encoding/utf8_no_BOM_no_charset_attr.css
A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
A test/css/encoding/utf16LE_with_BOM.lzx
A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
A test/css/encoding/iso8859-1_with_charset_attr.css
A test/css/encoding/utf16BE_with_BOM.css
A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
M WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
M WEB-INF/lps/server/src/org/openlaszlo/compiler/
StyleSheetCompiler.java
Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
