It looks like a step in the right direction. I need to examine
compileScript in the impl to see if perhaps we cannot even abstract a
little bit more. Regardless, I suspect having a parse in
JavaEmbedUtils is probably right.
-Tom
On 10/29/07, Yoko Harada <[EMAIL PROTECTED]> wrote:
> According to the message posted to jruby-user by Tom, I created some
> methods that encapsulate internal parseXXX methods. I attached a diff
> of JavaEmbedUtils.java, but I'm not sure this works as a patch file
> because I compared with the original file in .svn directory. Since
> current version of parseXXX methods don't need RAW data, I didn't wrap
> Java String Object by JavaEmbedUtils.javaToRuby().
>
> New version of JRubyScriptEngine.java that uses new encapsulated
> methods would be like this:
>
> private synchronized Node compileScript(String script, ScriptContext ctx)
> throws ScriptException {
> GlobalVariables oldGlobals = runtime.getGlobalVariables();
> try {
> setGlobalVariables(ctx);
> String filename = (String)
> ctx.getAttribute(ScriptEngine.FILENAME);
> if (filename == null) {
> filename = "<unknown>";
> }
> return JavaEmbedUtils.parse(runtime, script, filename);
> } catch (Exception exp) {
> throw new ScriptException(exp);
> } finally {
> if (oldGlobals != null) {
> setGlobalVariables(oldGlobals);
> }
> }
> }
>
> private synchronized Node compileScript(Reader reader,
> ScriptContext ctx) throws ScriptException {
> GlobalVariables oldGlobals = runtime.getGlobalVariables();
> try {
> setGlobalVariables(ctx);
> String filename = (String)
> ctx.getAttribute(ScriptEngine.FILENAME);
> if (filename == null) {
> return JavaEmbedUtils.parse(runtime, reader);
> } else {
> return JavaEmbedUtils.parse(runtime, reader, filename,
> runtime.getCurrentContext().getCurrentScope());
> }
> } catch (IOException exp) {
> throw new ScriptException(exp);
> }
> }
>
> These changes worked correctly as I wanted to do when I tested JSR223
> API by executing small programs. Are these appropriate?
>
> -Yoko
>
> On 10/26/07, Thomas E Enebo <[EMAIL PROTECTED]> wrote:
> > Yes. So combine this with my other message about encapsulating bodies
> > of those methods into JavaEmbedUtils. This will also help our impl
> > of BSF.
> >
> > -Tom
> >
> > On 10/26/07, Yoko Harada <[EMAIL PROTECTED]> wrote:
> > > It seems that JSR 223 implementation made an unwanted path to parser.
> > > JSR223's JRubyScriptEngine gives encoded, not RAW string to parser.
> > > Sounds like a JRubyScriptEngine's problem, again.
> > >
> > > -Yoko
> > >
> > > On 10/26/07, Thomas E Enebo <[EMAIL PROTECTED]> wrote:
> > > > Incoming string to parser should be treated as RAW or ISO8859_1 so
> > > > that every single byte still gets read. The parser will not properly
> > > > read chars since Ruby only works at a byte level. Did that make
> > > > sense?
> > > >
> > > > -Tom
> > > >
> > > > On 10/26/07, Yoko Harada <[EMAIL PROTECTED]> wrote:
> > > > > Hi,
> > > > >
> > > > > While I was testing JSR223 API, I hit JRuby's i18n bug. After fixing
> > > > > problems in JSR223's JRubyScriptEngine, I tried to print Japanese
> > > > > characters. This ended up in Mojibake. Supposed bug resides in
> > > > > org.jruby.astStrNode and org.jruby.util.ByteList, both of them handle
> > > > > characters as byte array. ByteList.append(int) method invoked from
> > > > > StringTerm.parseStringIntoBuffer method casts 16 bits char to 8 bits
> > > > > byte. As a result, this method drops 8btis and causes Mojibake. Like
> > > > > other mutibyte languages, Japanese characters needs 16bits to express
> > > > > a single character, so byte type, which has only 8bits, is short for
> > > > > multibyte languages. In terms of i18n, StrNode should have a char
> > > > > based buffer rather than a ByteList type.
> > > > >
> > > > > But, if StrNode needs to keep a byte array, the i18n problem should be
> > > > > fixed by converting character correctly to byte array as in this code.
> > > > >
> > > > > import java.io.UnsupportedEncodingException;
> > > > > import java.nio.ByteBuffer;
> > > > > import java.nio.CharBuffer;
> > > > > import java.nio.charset.CharacterCodingException;
> > > > > import java.nio.charset.Charset;
> > > > > import java.nio.charset.CharsetEncoder;
> > > > > import java.util.logging.Level;
> > > > > import java.util.logging.Logger;
> > > > >
> > > > > public class CharacterTest {
> > > > >
> > > > > public static void main(String[] args) throws
> > > > > UnsupportedEncodingException {
> > > > > char[] hello = {'こ', 'ん', 'に', 'ち', 'は'}; //Japanese
> > > > > characters are here.
> > > > > String defaultEncodingName =
> > > > > System.getProperty("sun.jnu.encoding");
> > > > > for (char c : hello) {
> > > > > byte[] bytes = getByteArrayFromChar(c,
> > > > > defaultEncodingName);
> > > > > System.out.print(new String(bytes, defaultEncodingName));
> > > > > }
> > > > > }
> > > > >
> > > > > private static byte[] getByteArrayFromChar(char c, String
> > > > > encodingName) {
> > > > > try {
> > > > > CharsetEncoder encoder =
> > > > > Charset.forName(encodingName).newEncoder();
> > > > > CharBuffer cbuf = CharBuffer.allocate(1);
> > > > > cbuf.put(c);
> > > > > cbuf.flip();
> > > > > ByteBuffer buf = encoder.encode(cbuf);
> > > > > int nbytes = buf.limit();
> > > > > byte[] encodedBytes = new byte[nbytes];
> > > > > buf.get(encodedBytes);
> > > > > return encodedBytes;
> > > > > } catch (CharacterCodingException ex) {
> > > > >
> > > > > Logger.getLogger(CharacterTest3.class.getName()).log(Level.SEVERE,
> > > > > null, ex);
> > > > > }
> > > > > return null;
> > > > > }
> > > > > }
> > > > >
> > > > > -Yoko
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe from this list please visit:
> > > > >
> > > > > http://xircles.codehaus.org/manage_email
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Blog: http://www.bloglines.com/blog/ThomasEEnebo
> > > > Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe from this list please visit:
> > > >
> > > > http://xircles.codehaus.org/manage_email
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe from this list please visit:
> > >
> > > http://xircles.codehaus.org/manage_email
> > >
> > >
> >
> >
> > --
> > Blog: http://www.bloglines.com/blog/ThomasEEnebo
> > Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this list please visit:
> >
> > http://xircles.codehaus.org/manage_email
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
