rzo1 commented on code in PR #829:
URL: https://github.com/apache/opennlp/pull/829#discussion_r2248223227
##########
opennlp-core/opennlp-runtime/src/main/java/opennlp/tools/sentdetect/SentenceDetectorME.java:
##########
@@ -340,15 +340,27 @@ protected boolean isAcceptableBreak(CharSequence s, int
fromIndex, int candidate
return true;
for (StringList abb : abbDict) {
- String token = abb.getToken(0);
+ final String token = abb.getToken(0);
+ final int tokenPosition = s.toString().indexOf(token, fromIndex);
+ if (tokenPosition == -1) {
+ continue; // skip fast
+ }
+ final char prevChar = s.charAt(tokenPosition - 1);
int tokenLength = token.length();
- int tokenPosition = s.toString().indexOf(token, fromIndex);
- if (tokenPosition + tokenLength < candidateIndex || tokenPosition >
candidateIndex)
- continue;
+ if (tokenPosition + tokenLength < candidateIndex || tokenPosition >
candidateIndex ||
+ /*
+ * Note:
+ * Skip abbreviation candidate if regular characters exist directly
before it,
+ * That is, any letter or digit except: a whitespace, an apostrophe,
or an opening round bracket.
Review Comment:
But I didn't saw the escaped ', so all good. 😁🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@opennlp.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
