Kristian Rickert created OPENNLP-1887:
-----------------------------------------
Summary: Lexical expansion over the knowledge base seam: synonyms,
hypernyms, optional hyponyms
Key: OPENNLP-1887
URL: https://issues.apache.org/jira/browse/OPENNLP-1887
Project: OpenNLP
Issue Type: New Feature
Components: Sentence Detector
Reporter: Kristian Rickert
Assignee: Kristian Rickert
STACKED on [#1155|https://github.com/apache/opennlp/pull/1155] (wordnet-api):
this branch builds on that PR's LexicalKnowledgeBase seam, and since that head
lives on a fork it cannot be this PR's base ref. Until
[#1155|https://github.com/apache/opennlp/pull/1155] merges, the diff here shows
its commits too; only the last commit
([{{a000fcc}}|https://github.com/apache/opennlp/commit/a000fcce4959ed4081ca1234c323e957b2e20801])
is this PR. After [#1155|https://github.com/apache/opennlp/pull/1155] lands,
the branch rebases onto main and the diff collapses to this change alone.
Adds LexicalExpander to the opennlp-wordnet module: given a term, it produces
weighted related terms from a LexicalKnowledgeBase, the synonyms sharing its
synsets, the lemmas of hypernym ancestors up to a configured depth (following
both the direct and the instance relation, visited-checked so cyclic data
terminates), and optionally direct hyponyms. Each expansion carries a
deterministic heuristic weight (sense-rank decay times depth decay, both
configurable), results are deduplicated case-insensitively keeping the highest
weight, the input term is never returned, and ordering is stable. An optional
Lemmatizer fallback expands inflected input; the Morphy lemmatizer of this
module plugs in directly, so dogs expands via dog and mice via mouse.
Tests cover the behavior over a hand-built graph with a controlled shape (sense
ranking, depth decay, dedupe, cycle termination, hyponym opt-in, configuration
validation) plus integration over the miniature WN-LMF and WNDB fixtures
asserting both readers expand identically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
