krickert commented on PR #1161: URL: https://github.com/apache/opennlp/pull/1161#issuecomment-4940070596
Thanks. Addressed in 2afe0b4e: - CodePoints public API: reduced to package-private in the normalizer package. The tokenizer and Confusables hot loops went back to plain Character.codePointAt/charCount decoding, so the class is a package-local detail of the fold loops, not compatibility-bearing surface, and the cross-package placement question is moot. The one remaining public addition is Alignment.Builder(int), which is used from opennlp-dl and opennlp-runtime and is deliberate additive API: a pre-sizing overload beside the existing public no-arg builder. - MemberFold megamorphic risk: the lambda indirection is gone. normalize, collapse, and removeAll each carry their own primitive loop behind the shared first-member scan, so there is no shared virtual call site left to go megamorphic and no per-code-point record in any fold loop. Output remains byte-identical; the identity short-circuits, the Confusables prescan, and the pre-sized collections are unchanged. - CodePoints test coverage: CodePointsTest added, checking at() and before() against Character.codePointAt/codePointBefore over BMP text, a paired supplementary character, a lone high surrogate, and a lone low surrogate, which covers the previously unverified low-surrogate branch. - At index asymmetry: gone with the visibility reduction and the hot-loop revert; the record no longer travels across packages. - Record construction in the tokenizer's hottest loops: reverted to primitives, so there is nothing left to confirm by benchmark. - ignoredIndex misnomer: gone with the MemberFold removal. - Confusables local shadowing the static field: renamed to d, matching the accessor. - Aligned digit fold: a normalizeAligned test over mixed ASCII and non-ASCII digits now pins the span mapping, so the equal-run treatment of ASCII digits is held by a test rather than a comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
