krickert opened a new pull request, #1155: URL: https://github.com/apache/opennlp/pull/1155
Adds the format-agnostic lexical knowledge base seam to opennlp-api (`opennlp.tools.wordnet`: `LexicalKnowledgeBase`, `Synset`, `WordNetPos`, `WordNetRelation`), and a new `opennlp-extensions/opennlp-wordnet` module with two clean-room readers behind it: - `WnLmfReader`: a StAX reader for WN-LMF XML, Open English WordNet's interchange format. XXE-hardened per the OWASP posture for DOCTYPE-bearing formats: the DOCTYPE line real OEWN releases ship is tokenized and skipped while external entities and the external DTD subset stay fully disabled, so an unmodified download parses directly; a dedicated test proves a DOCTYPE-declared internal-subset entity payload fails loud rather than expanding. - `WndbReader`: a reader for legacy Princeton WNDB directories, validating the format's documented offset contract. Both produce equivalent immutable, thread-safe lexicon views, pinned by a structural equivalence test over matching miniature fixtures. On top of the seam, `MorphyLemmatizer` implements the existing `Lemmatizer` interface with the documented Morphy algorithm: exception-list lookup first, then per-POS detachment rules with every candidate validated against the lexicon. **Naming:** the seam interface is `LexicalKnowledgeBase` rather than a WordNet-branded name; the contract is generic and only the format-specific readers carry the WordNet name. The module ships code only; users point it at a WordNet database they downloaded. Verified against Princeton 3.0 (117,659 synsets) and OEWN 2024 (120,630 synsets). No third-party WordNet library is used or referenced. Verification: api 303/0 (+17), opennlp-wordnet 86/0, full reactor verify green. Small upstream note found along the way: `DictionaryLemmatizer` javadoc says unknown = "0" but the code returns the letter "O"; Morphy matches the code. Filed separately. https://issues.apache.org/jira/browse/OPENNLP-1880 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
