[
https://issues.apache.org/jira/browse/OPENNLP-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054372#comment-13054372
]
Nicolas Hernandez commented on OPENNLP-203:
-------------------------------------------
As shown in my previous file, I filtered the lines which do not end with an
assumed end sentence character.
Anyway I tested a model trained on 1 million of europarl sentences.
In my text test, I have one 'M.' ('Mr.' in French) in the middle of one
sentence and two occurences as the first token of a sentence.
The two occurrences begining a sentence are wrongly split but not the other
one.
I do not infer anything but I note it partially works.
'M.' occurs 58 times in the training corpus.
> UIMA Sentence Detector Trainer builds models which do not split correctly the
> sentences
> ---------------------------------------------------------------------------------------
>
> Key: OPENNLP-203
> URL: https://issues.apache.org/jira/browse/OPENNLP-203
> Project: OpenNLP
> Issue Type: Bug
> Components: Sentence Detector, UIMA Integration
> Affects Versions: tools-1.5.1-incubating
> Environment: OS
> Linux version 2.6.32-30-generic (buildd@vernadsky) (gcc version 4.4.3 (Ubuntu
> 4.4.3-4ubuntu5) ) #59-Ubuntu SMP Tue Mar 1 21:30:21 UTC 2011
> JVM
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
> Reporter: Nicolas Hernandez
> Fix For: tools-1.5.2-incubating
>
>
> The models trained with the UIMA component give wrong begin/end offset
> despite the fact they manage to split text in sentences.
> I observed that the begin of a current sentence starts including as a first
> token the punctuation character of the previous one while the
> previous one does not include it as its last one.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
