[
https://issues.apache.org/jira/browse/OPENNLP-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204690#comment-13204690
]
William Colen commented on OPENNLP-428:
---------------------------------------
Thank you very much for the patch! Looks good. All tests are passing after
applying it.
Just a few comments:
You are passing the EOS characters in a string, and using a simple
String.toCharArray() to get the chars. It is simple and should work for most of
the languages. But won't work for Thai for example, because we can't pass a
space and a new line using the command line tool.
SentenceDetectorME.java
- We should keep the old train method, we can't break backward compatibility.
Create a new train method with the new argument.
- We don't need to modify the deprecated train methods.
BaseModel.java
- Remove the unnecessary sysout.
SDContextGenerator.java
- Keep the old constructor, we should be backward compatible. Add a new
constructor with the new argument.
> make EOS character set configurable
> -----------------------------------
>
> Key: OPENNLP-428
> URL: https://issues.apache.org/jira/browse/OPENNLP-428
> Project: OpenNLP
> Issue Type: Improvement
> Components: Sentence Detector
> Reporter: Katrin Tomanek
> Priority: Minor
> Fix For: tools-1.5.3-incubating
>
> Attachments: patch_eos_characters
>
>
> Currently, the EOS symbols to be used by the sentence detector cannot be
> configured (at the moment, a user would have to make changes in
> opennlp.tools.sentdetect.lang.Factory
> Since it is important to use the same EOS symbols during training and during
> testing/prediction, the EOS symbols should be stored with the model's
> properties
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
