[
https://issues.apache.org/jira/browse/LUCENE-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325674#comment-14325674
]
Dawid Weiss commented on LUCENE-6254:
-------------------------------------
Lemmatisation is a tricky thing, especially for highly inflectional languages.
There are technical issues (the dictionaries can get quite big; that's why
morfologik-stemming uses an automaton to encode it efficiently) and
non-technical issues (lemmatisation is typically combined with morfphological
analysis to resolve disambiguities, otherwise it's not clear which lemma to
pick for ambiguous surface forms).
> Dictionary-based lemmatizer
> ---------------------------
>
> Key: LUCENE-6254
> URL: https://issues.apache.org/jira/browse/LUCENE-6254
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/analysis
> Reporter: Erlend Garåsen
> Labels: patch
> Fix For: 5.0
>
> Attachments: LUCENE-6254.patch
>
>
> The only way to achieve lemmatization today is to use the
> SynonymFilterFactory. The available stemmers are also inaccurate since they
> are only following simplistic rules.
> A dictionary-based lemmatizer will be more precise because it has the
> opportunity to know the part of speech. Thus it provides a more precise
> method to stem words compared to other dictionary-based stemmers such as
> Hunspell.
> This is my effort to develop such a lemmatizer for Apache Lucene. The
> documentation is temporarily placed here:
> http://folk.uio.no/erlendfg/solr/lemmatizer.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
