krickert commented on PR #1150: URL: https://github.com/apache/opennlp/pull/1150#issuecomment-4959066625
Also - the eval test did fail. On exactly this :) So I updated the training-data fingerprint in SourceForgeModelEval to match, since SimpleTokenizer now splits on the Unicode `White_Space` set. The real blast radius is small: 64 NEL and 2 C0-separator characters across the 300K-line Leipzig corpus retokenize, and every accuracy eval (NameFinder, POS, parser, chunker, sentence detector) passed unchanged. Since 3.0 is a new major, I imagine that it's normal for packages to standardize on one whitespace definition than carry the legacy behavior forward, and existing models still load and score the same features because the feature generator keeps the legacy predicate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
