According to the message posted to jruby-user by Tom, I created some
methods that encapsulate internal parseXXX methods. I attached a diff
of JavaEmbedUtils.java, but I'm not sure this works as a patch file
because I compared with the original file in .svn directory. Since
current version of parseXXX methods don't need RAW data, I didn't wrap
Java String Object by JavaEmbedUtils.javaToRuby().
New version of JRubyScriptEngine.java that uses new encapsulated
methods would be like this:
private synchronized Node compileScript(String script, ScriptContext ctx)
throws ScriptException {
GlobalVariables oldGlobals = runtime.getGlobalVariables();
try {
setGlobalVariables(ctx);
String filename = (String) ctx.getAttribute(ScriptEngine.FILENAME);
if (filename == null) {
filename = "<unknown>";
}
return JavaEmbedUtils.parse(runtime, script, filename);
} catch (Exception exp) {
throw new ScriptException(exp);
} finally {
if (oldGlobals != null) {
setGlobalVariables(oldGlobals);
}
}
}
private synchronized Node compileScript(Reader reader,
ScriptContext ctx) throws ScriptException {
GlobalVariables oldGlobals = runtime.getGlobalVariables();
try {
setGlobalVariables(ctx);
String filename = (String) ctx.getAttribute(ScriptEngine.FILENAME);
if (filename == null) {
return JavaEmbedUtils.parse(runtime, reader);
} else {
return JavaEmbedUtils.parse(runtime, reader, filename,
runtime.getCurrentContext().getCurrentScope());
}
} catch (IOException exp) {
throw new ScriptException(exp);
}
}
These changes worked correctly as I wanted to do when I tested JSR223
API by executing small programs. Are these appropriate?
-Yoko
On 10/26/07, Thomas E Enebo <[EMAIL PROTECTED]> wrote:
> Yes. So combine this with my other message about encapsulating bodies
> of those methods into JavaEmbedUtils. This will also help our impl
> of BSF.
>
> -Tom
>
> On 10/26/07, Yoko Harada <[EMAIL PROTECTED]> wrote:
> > It seems that JSR 223 implementation made an unwanted path to parser.
> > JSR223's JRubyScriptEngine gives encoded, not RAW string to parser.
> > Sounds like a JRubyScriptEngine's problem, again.
> >
> > -Yoko
> >
> > On 10/26/07, Thomas E Enebo <[EMAIL PROTECTED]> wrote:
> > > Incoming string to parser should be treated as RAW or ISO8859_1 so
> > > that every single byte still gets read. The parser will not properly
> > > read chars since Ruby only works at a byte level. Did that make
> > > sense?
> > >
> > > -Tom
> > >
> > > On 10/26/07, Yoko Harada <[EMAIL PROTECTED]> wrote:
> > > > Hi,
> > > >
> > > > While I was testing JSR223 API, I hit JRuby's i18n bug. After fixing
> > > > problems in JSR223's JRubyScriptEngine, I tried to print Japanese
> > > > characters. This ended up in Mojibake. Supposed bug resides in
> > > > org.jruby.astStrNode and org.jruby.util.ByteList, both of them handle
> > > > characters as byte array. ByteList.append(int) method invoked from
> > > > StringTerm.parseStringIntoBuffer method casts 16 bits char to 8 bits
> > > > byte. As a result, this method drops 8btis and causes Mojibake. Like
> > > > other mutibyte languages, Japanese characters needs 16bits to express
> > > > a single character, so byte type, which has only 8bits, is short for
> > > > multibyte languages. In terms of i18n, StrNode should have a char
> > > > based buffer rather than a ByteList type.
> > > >
> > > > But, if StrNode needs to keep a byte array, the i18n problem should be
> > > > fixed by converting character correctly to byte array as in this code.
> > > >
> > > > import java.io.UnsupportedEncodingException;
> > > > import java.nio.ByteBuffer;
> > > > import java.nio.CharBuffer;
> > > > import java.nio.charset.CharacterCodingException;
> > > > import java.nio.charset.Charset;
> > > > import java.nio.charset.CharsetEncoder;
> > > > import java.util.logging.Level;
> > > > import java.util.logging.Logger;
> > > >
> > > > public class CharacterTest {
> > > >
> > > > public static void main(String[] args) throws
> > > > UnsupportedEncodingException {
> > > > char[] hello = {'こ', 'ん', 'に', 'ち', 'は'}; //Japanese
> > > > characters are here.
> > > > String defaultEncodingName =
> > > > System.getProperty("sun.jnu.encoding");
> > > > for (char c : hello) {
> > > > byte[] bytes = getByteArrayFromChar(c, defaultEncodingName);
> > > > System.out.print(new String(bytes, defaultEncodingName));
> > > > }
> > > > }
> > > >
> > > > private static byte[] getByteArrayFromChar(char c, String
> > > > encodingName) {
> > > > try {
> > > > CharsetEncoder encoder =
> > > > Charset.forName(encodingName).newEncoder();
> > > > CharBuffer cbuf = CharBuffer.allocate(1);
> > > > cbuf.put(c);
> > > > cbuf.flip();
> > > > ByteBuffer buf = encoder.encode(cbuf);
> > > > int nbytes = buf.limit();
> > > > byte[] encodedBytes = new byte[nbytes];
> > > > buf.get(encodedBytes);
> > > > return encodedBytes;
> > > > } catch (CharacterCodingException ex) {
> > > >
> > > > Logger.getLogger(CharacterTest3.class.getName()).log(Level.SEVERE,
> > > > null, ex);
> > > > }
> > > > return null;
> > > > }
> > > > }
> > > >
> > > > -Yoko
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe from this list please visit:
> > > >
> > > > http://xircles.codehaus.org/manage_email
> > > >
> > > >
> > >
> > >
> > > --
> > > Blog: http://www.bloglines.com/blog/ThomasEEnebo
> > > Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe from this list please visit:
> > >
> > > http://xircles.codehaus.org/manage_email
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this list please visit:
> >
> > http://xircles.codehaus.org/manage_email
> >
> >
>
>
> --
> Blog: http://www.bloglines.com/blog/ThomasEEnebo
> Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
