[
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567775#comment-13567775
]
Kai Gülzau edited comment on LUCENE-2899 at 1/31/13 7:11 PM:
-------------------------------------------------------------
End of OpenNLPTokenizer.fillBuffer() should be:
{code}
while(length == size) {
offset += size;
fullText = Arrays.copyOf(fullText, offset + size);
length = input.read(fullText, offset, size);
}
if (length == -1) {
length = 0;
}
fullText = Arrays.copyOf(fullText, offset + length);
{code}
was (Author: [email protected]):
End of OpenNLPTokenizer.fillBuffer() should be:
{code}
if (length > 0) {
fullText = Arrays.copyOf(fullText, offset + length);
}
{code}
> Add OpenNLP Analysis capabilities as a module
> ---------------------------------------------
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/analysis
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch,
> LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java,
> OpenNLPTokenizer.java, opennlp_trunk.patch
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice
> to have a submodule (under analysis) that exposed capabilities for it. Drew
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
