krickert opened a new pull request, #1161: URL: https://github.com/apache/opennlp/pull/1161
## Summary Reopens the perf work from closed draft #1153, now based cleanly on `main` after #1105 merged. Non-breaking, output-preserving optimizations for heavy analyzer/tokenizer use: - **`CodePoints`** — shared BMP-aware decode helper (replaces repeated inline surrogate blocks) - **`CharClass`** — identity short-circuit when text has no member code points; shared `foldUnlessClean` for normalize/collapse/removeAll - **`Confusables.skeleton`** — BitSet key pre-filter + fast path for clean ASCII / already-NFD text (skips both NFD passes) - **`Alignment.Builder(int)`** — pre-sized backing arrays from expected text length - **DL components** — `ArrayList` / `HashMap.newHashMap(3)` sized up front instead of `LinkedList` / default `HashMap` No docbook or API changes. Full 1850 test suites pass unchanged. ## JMH (mixed-script token streams) | Optimization | Result | |---|---| | Identity short-circuit (whitespace normalize, member-free text) | up to **~4.3×** | | Confusables skeleton (clean ASCII) | **~1.7×** | | Fold-saturated text | neutral (expected) | ## Test plan - [x] `CharClassTest` (43), `AlignmentTest`, `ConfusablesTest` (13), `SetBasedNormalizerTest` (15), `WordSegmenterTest` (14) - [ ] Full `opennlp-api`, `opennlp-runtime`, `opennlp-dl` suites on CI https://issues.apache.org/jira/browse/OPENNLP-1878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
