Markus Klose created SOLR-3630:
----------------------------------
Summary: latin text analysis
Key: SOLR-3630
URL: https://issues.apache.org/jira/browse/SOLR-3630
Project: Solr
Issue Type: New Feature
Components: Schema and Analysis
Reporter: Markus Klose
Priority: Minor
Hi
a workmate and I played a bit with latin text analysis and created two filter
for the solr trunk version.
One filter is designed for number conversion like 'iv' -> '4', 'v' -> '5', 'vi'
-> '6' ...
The second filter is a stemmer for the most common suffixe.
The following schema configuration could be a usecase for latin stemming.
<fieldType name="text_latin" class="solr.TextField"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter
class="org.apache.solr.analysis.LatinNumberConvertFilterFactory"
strictMode="true"/>
<filter class="solr.KeywordMarkerFilterFactory"
protected="latin_protwords.txt" />
<filter
class="org.apache.solr.analysis.LatinStemFilterFactory" />
</analyzer>
</fieldType>
LatinNumberConvertFilterFactory has one property "strictMode" (default is
false). This boolean indicates in which way the computation of the value is
done, because not all letter combination are "valid" numbers. With
strictMode="true" the output of "ic" is "ic"; With strictMode="false" the
output of "ic" is "99"
The LatinStemFilterFactory generates for each input token two output token. the
first stemmed as noun and the second stemmed as verb.
Both filter are aware of the KeywordMarkerFilterFactory.
I have attached the svn patch for both filter. In addition I attached to zip
files that are needed by filter tests (TestLatinNumberConvertFilter,
TestLatinStemFilter). I am sorry for that but i did not find the option to
include them into the patch, if there is one.
The image latin_analysis.png is an example of the analysis done with the
configuration above. For this test we used the jar file latin.analysis.jar
Have fun with latin text analysis.
It would be great to get some feedback.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
