On Sep 3, 2008, at 1:31 PM, Johannes Luber wrote: > Terence Parr schrieb: >> Hi, I designed everything to be 32 bit clean in terms of token types >> and character input so, while \uFFFF is not about character, there is >> no reason we can't allow that is input. currently we do not. I set >> the >> maximum to \uFFFE but I am changing it to: >> >> public static final int MAX_CHAR_VALUE = '\uFFFF'; >> >> My unit tests and examples directory seemed to work okay. The Java.g >> grammar for Sun needs to mimic what the javac compiler does; it allow >> us '\uFFFF' and more importantly converts that to the single Unicode >> character code point BEFORE the compiler sees it. it is done in the >> character string. anyway, ANTLR says that is an invalid character at >> the moment. I don't think we will have a problem... can anyone think >> of an issue? I do all of my checks using -1 not '\uFFFF' for EOF...we >> *should* be okay... >> >> Ter > > Does unicode restrict the use \uffff? Will you sometime add handling > for > chars bigger than \uffff natively in ANTLR, even if you need some > translation functions to make things work in JAva?
I haveTo figure out what all the 32bit character stuff means... The machinery of ANTLR should be able to handle 32-bit signed integers as characters or token types... representing them in a Java string for the Java target is another matter ;) Ter _______________________________________________ antlr-dev mailing list [email protected] http://www.antlr.org:8080/mailman/listinfo/antlr-dev
