krickert opened a new pull request, #1153: URL: https://github.com/apache/opennlp/pull/1153
Draft, stacked on `OPENNLP-1850-3-dl` (PR #1105): this base is deliberate, not a mistake. #1105 is approved but not yet merged, and the eval-tests-configurable job that validates it is still being worked through; opening this PR against `main` right now would bundle all of #1105's unmerged commits into the diff alongside these 6. Once #1105 merges, this PR gets retargeted to `main` and the branch rebased, at which point the diff naturally collapses to just the 6 commits below. Non-breaking performance follow-up to OPENNLP-1850 for heavy analyzer/tokenizer use. Six output-preserving changes: - an identity short-circuit on the non-aligned `CharClass` folds (member-free text returns without allocation), - a key-`BitSet` fast-path in `Confusables.skeleton` (clean text skips both NFD passes), - BMP fast-paths in the hot cursor loops, - precomputed digit strings, - a pre-sized `Alignment.Builder` constructor, - up-front sizing of the DL score/token/ONNX collections. JMH on realistic mixed-script token streams confirms the tier-one wins (identity short-circuit up to 4.3x on whitespace-normalize-heavy streams, Confusables 1.7x) with the predicted neutral result on fold-saturated text. Byte-identical output; the full 1850 suites pass unchanged (api 298/0, runtime 1304/0, dl 92/0). https://issues.apache.org/jira/browse/OPENNLP-1878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
