Hi guys,
I'm pleased to see velocity 1.1-rc1 released. It can specify template-based encoding
method, which is very useful to internatinal applications. Unfortunitely, there's
still a bug(i think, maybe it's not velocity's bug, it's about JavaCC). How can
velocity process international characters such as Chinese? I modified the
input.encoding=GBK property in velocity.properties file, to make it recogonize
Chinese. It seems work well, until it saw a Chinese character (U+4e0a). Velocity
complained it encountered a "\n" after double-quote. Ya, it's velocity(javacc) masks
the higher byte, so it considered (U+4e0a) as (U+000a), which is same as "\n".
I tried to correct this by adding a new line "UNICODE_INPUT=true" in
"org/apache/velocity/runtime/parser/Parser.jjt" file, and rebuild the velocity. So
that JavaCC generates "UCode_CharStream.java" instead of "ASCII_CharStream.java". But
the result becomes more strange! Because javacc eats every 2 characters and combines
them as one character regardless of you initialize the parser by a byte-based stream
or character-based reader! Finally, I found the solution.
* First, add option "UNICODE_INPUT=true" to Parser.jjt, and execute the "build" shell
in "/org/apache/velocity/runtime/parser" directory. Then it will generate a file
UCode_CharStream.java.
* Replace the UCode_CharStream.java with original ASCII_CharStream.java by shell
command: mv ASCII_CharStream.java UCode_CharStream.java
* Modify the UCode_CharStream.java by vi UCode_CharStream.java
1. replace the class name and constructor names with UCode_CharStream.
2. modify lines below in the method readChar(), so that it can process UNICODE
correctly.
change: return (char)((char)0xff & buffer[(bufpos == bufsize - 1) ? (bufpos =
0) : ++bufpos]);
to: return buffer[(bufpos == bufsize - 1) ? (bufpos = 0) : ++bufpos];
change: char c = (char)((char)0xff & buffer[bufpos]);
to: char c = buffer[bufpos];
* Use of "USER_CHAR_STREAM=true" in javacc is also work.
Now i think the product is perfect!
Michael Zhou
