Katakana characters in queries (a bug?)

[Geir Ove Grønmo]

Hi!

There seem to be a bug in the lucene-1.2-rc1.jar distribution. Searching
for the following string returns an error message from the query parser.

String katakana = "\u30AB\u30BF\u30AB\u30CA";

- - - 
org.apache.lucene.queryParser.TokenMgrError: Lexical error at line 1, column 10.  
Encountered: "\u00ab" (171), after : ""
        at org.apache.lucene.queryParser.QueryParserTokenManager.getNextToken(Unknown 
Source)
        at org.apache.lucene.queryParser.QueryParser.jj_ntk(Unknown Source)
        at org.apache.lucene.queryParser.QueryParser.Clause(Unknown Source)
        at org.apache.lucene.queryParser.QueryParser.Query(Unknown Source)
        at org.apache.lucene.queryParser.QueryParser.parse(Unknown Source)
        at org.apache.lucene.queryParser.QueryParser.parse(Unknown Source)
        ...
- - -

This query used to work in the 1.0 release.

All the best,
Geir O.