Tucker, Wanli,
I've done some testing with OL 4.0.15 and Webtop running 1.5.2
running on
OL 4.0.15. While the fix works for OL trunk, it doesn't work for
4.0.15.
Here's the debug output for test file
"test/css/encoding/utf8_with_BOM_no_charset_attr.lzx". The UTF-8
encoded
data is read correctly.
http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler -
StyleSheetCompiler.compile called!
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler -
@charset=utf-8 found on stylesheet tag
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
compiler.StyleSheetCompiler -
reading in stylesheet from src="utf8_with_BOM_no_charset_attr.css"
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/
src/
svn/openlaszlo/4.0.15/test/css/encoding/
utf8_with_BOM_no_charset_attr.css
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
addFile
Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/
encoding/utf8_with_BOM_no_charset_attr.css
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
creating CSSHandler
09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
Trying
to parse CSS with charset setting of utf-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Read
the following bytes: text[name='german'] {
backgroundColor: #ffcc00;
buttonText: "öäüÖÄܧ";
}
text[nam
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Testing
for UTF-8 BOM!
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
Found
BOM on file, encoding is UTF-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
Opening
CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
css/encoding/utf8_with_BOM_no_charset_attr.css using encoding
UTF-8
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
Skip
first 3 bytes containing UTF-8 BOM
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
compiling CSSHandler using new unique names
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Conditional selector: [name="german"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Cond string: { attrname: "name", attrvalue: "german",
simpleselector:
"text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Conditional selector: [name="korean"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Cond string: { attrname: "name", attrvalue: "korean",
simpleselector:
"text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Conditional selector: [name="chinese"]
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Attribute condition
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
simple selector:text
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
Cond string: { attrname: "name", attrvalue: "chinese",
simpleselector:
"text"}
09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
compiler.StyleSheetCompiler -
whole stylesheet as css $lzc $style._addRule(new $lzc
$rule({ attrname:
"name", attrvalue: "german", simpleselector: "text"},
{backgroundColor:
0xFFCC00, buttonText: "???????"}));
$lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
"korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
buttonText:
"? ?? ??? ?? ?? ?? ?? ??"}));
$lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
"chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
buttonText:
"???????????????"}))
But the debug console in the browser shows the following error
message:
ERROR: Unknown cohort for rule: #0 * (undefined)
ERROR: Unknown cohort for rule: #1 * (undefined)
ERROR: Unknown cohort for rule: #2 * (undefined)
WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
for
node «lz.text#3| #german» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
for
node «lz.text#3| #german» for property name backgroundColor
WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
for
node «lz.text#4| #korean» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
for
node «lz.text#4| #korean» for property name backgroundColor
WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
for
node «lz.text#5| #chinese» for property name buttonText
WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
for
node «lz.text#5| #chinese» for property name backgroundColor
Any idea what this might be?
- Raju
On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
[cc-ing Wanli]
Wanli,
Maybe you also want to review (or have someone on your team
review) as
this issue affects you?
---
Raju,
This looks good.
I think you might have trouble trying checking in your utf-16
examples
to svn. I'm not sure svn will do the right thing. It looks
like it is
treating them as binary files. Maybe that will work. I don't
know.
It looks to me like you may also have some tab characters in your
sources. You need to expand tabs to check in, otherwise the
pre- commit
filter will reject your checkin.
I approve this change. You might want to check in your fix
separately
from your tests, just in case the UTF-16 files cause svn issues.
On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
Added a "+" in debug output string concatenation.
Change 20090608-raju-Y by [email protected]
on
2009-06-08 20:09:02 CEST
in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
for http://[email protected]/openlaszlo/trunk
Summary: Fix for CSS parser uses incorrect file encoding
New Features: Adds an optional @charset to the stylesheet tag,
in case
the user wants to use a CSS file in a different encoding then
utf-8
Bugs Fixed: LPP-8045
Technical Reviewer: ptw
QA Reviewer: (pending)
Doc Reviewer: (pending)
Documentation:
Release Notes:
Details:
+ StyleSheetCompiler.java: Handling for @charset on stylesheet
tag
added. Added a
2nd parameter with the encoding value to the CSSHandler.parse()
call.
+ CSSHandler.java:
parse() method takes the encoding from the LZX stylesheet tag
as a 2nd
parameter.
getInputSource() method does a few more things now:
1) checks for a possible BOM on the CSS file
2) checks if a possible BOM conflicts with the value of the
stylesheet
tag's @charset
3) if there's a BOM, the BOM bytes are removed from the input
stream
+ FileUtils.java:
Added method public static String
detectBOMEncoding(BufferedInputStream
in)
The method returns the BOM marker interpreted as one of the
following
strings:
UTF-8
UTF-16LE
UTF-16BE
Tests:
+ test files in folder test/css/encoding
The following test exist:
1) iso8859-1_with_charset_attr.lzx
Reading an iso-8859-2 encoded CSS file with some German
special chars
2) utf16BE_with_BOM.lzx
Reading an utf-16 BE CSS file with BOM marker
3) utf16LE_with_BOM.lzx
Reading an utf-16 LE CSS file with BOM marker
4) utf8_with_BOM_no_charset_attr.lzx
Reading an utf-8 CSS file with BOM and no charset attribute on
the
stylesheet tag
5) utf8_with_BOM_conflicting_charset_attr.lzx
Reading a CSS with @charset value of utf-16, but CSS having a
UTF-8
BOM marker, will
throw a compile error
Files:
A test/css/encoding
A test/css/encoding/iso8859-1_with_charset_attr.lzx
A test/css/encoding/utf16BE_with_BOM.lzx
A test/css/encoding/utf8_with_BOM_no_charset_attr.css
A test/css/encoding/utf16LE_with_BOM.css
A test/css/encoding/utf8_no_BOM_no_charset_attr.css
A test/css/encoding/
utf8_with_BOM_conflicting_charset_attr.css
A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
A test/css/encoding/utf16LE_with_BOM.lzx
A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
A test/css/encoding/iso8859-1_with_charset_attr.css
A test/css/encoding/utf16BE_with_BOM.css
A test/css/encoding/
utf8_with_BOM_conflicting_charset_attr.lzx
M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
M WEB-INF/lps/server/src/org/openlaszlo/utils/
FileUtils.java
M WEB-INF/lps/server/src/org/openlaszlo/compiler/
StyleSheetCompiler.java
Changeset:
http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
