[
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399245#comment-13399245
]
Lance Norskog commented on LUCENE-2899:
---------------------------------------
General status:
* At this point you have to download 1 library (jwnl) and run a script to make
the unit tests work.
* You have to download several model files from sourceforge to do real work.
There is no script to help.
* The tokenizer and filter are in solr/ not lucene/
What is missing to make this a full package:
* Payload handling
** TokenFilter to parse TAG/term or term_TAG into term/payload.
** Output code in Solr for the reverse.
** Payload query for tags.
** Similarity scoring algorithms for tags.
* Tag handling
** There is a universal set of 12 parts-of-speech tags, with mappings for many
language tagsets (Treebank etc.) into 12 common tags. Multi-language sites
would benefit from this. I persuaded the authors to switch from GNU to Apache
licensing.
*** [A Universal Part-of-Speech Tagset|http://arxiv.org/abs/1104.2086]
What NLP apps would be useful for search? Coordinate expansion, for example.
> Add OpenNLP Analysis capabilities as a module
> ---------------------------------------------
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Java
> Issue Type: New Feature
> Components: modules/analysis
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Attachments: LUCENE-2899.patch, opennlp_trunk.patch
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice
> to have a submodule (under analysis) that exposed capabilities for it. Drew
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
