Cédric Damioli wrote:
Hi all,
I noticed that no Lucene Analyzer is used when querying the repository :
when building the actual Lucene query the
o.a.j.c.query.lucene.LuceneQueryBuilder does not make any use of the
Analyzer (at least in my case).
in general the analyzer is used for the contains() function to tokenize
the fulltext query parameter. however there is one exception to this
rule: terms that use wildcards are not tokenized.
the reason for this is a technical one. an analyzer that is based on a
grammer will not be able to process such tokens properly.
e.g. if the grammar rule says 'a' 'b' and 'abc' are tokens then the
analyzer would be unable to determine if 'ab*' should be tokenized or not.
Let describe my exemple : I'm using chinese characters, say A and B. I
set a property named "title" with the value "AB" (the two chinese
characters without any witespace).
After indexation (with the default StandardAnalyzer) the text has been
tokenized and the index contains at least three noticeable terms :
- one associated with the field _PROPERTIES and the value "titleï¿¿AB"
- one associated with the field FULL:title and the value "A"
- one associated with the field FULL:title and the value "B"
After that I try to execute an XPath Query like //*[jcr:contains(@title,
'*AB*')]
I of course expected this query to return the previously set property,
but I obtained no results.
After looking at the code, I can say that the Analyzer is not called for
a WildcardQuery, so my "AB" is not tokenized and furthermore,
if you execute the following query you will get the expected result:
//*[jcr:contains(@title, 'AB')]
assuming A and B are chinese characters, they will get tokenized and the
fulltext query is acutally a phrase match. similar to searching for
'hello there'.
it seems
that the _PROPERTIES field is not used when searching, otherwise, I
think it would match.
the PROPERTIES field is only used for jcr:like and other operators.
e.g. you can search the workspace with the following query:
//*[jcr:like(@title, '%AB%')]
this will internally use the PROPERTIES field.
I know that StandardAnalyzer is not the best suited for handling chinese
text, but that's another story.
there might be implementations that are better suited for chinese text,
but I think it does a pretty good job.
It seems to me that there may be a Jackrabbit problem here, so I wanted
to have your feelings about this.
What you described is imo expected behaviour in jackrabbit.
Regarding analyzers, you can configure it on a per workspace basis and
use one of the many available analyzers. e.g. from the lucene website.
regards
marcel
