[
https://issues.apache.org/jira/browse/OPENNLP-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Wiesner updated OPENNLP-1870:
------------------------------------
Summary: Add emoji annotation layer wired to Sentiment, Name Finder, and
Doccat (was: Emoji annotation layer wired to Sentiment, Name Finder, and
Doccat)
> Add emoji annotation layer wired to Sentiment, Name Finder, and Doccat
> ----------------------------------------------------------------------
>
> Key: OPENNLP-1870
> URL: https://issues.apache.org/jira/browse/OPENNLP-1870
> Project: OpenNLP
> Issue Type: Task
> Components: dl, Sentiment Detection
> Affects Versions: 3.0.0-M4
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
>
> *Stacks on:* OPENNLP-1869.
> *Summary.* Extend the emoji lookup with a small, provenance-tagged annotation
> layer per symbol: human-readable name (CLDR short-name), sentiment (with its
> own {{sentiment_source}}), coarse entity type (VEHICLE, LANDMARK, ANIMAL,
> FLAG, FOOD, ...), and a document-category hint. Surface these as parallel
> layers on the {{Term}} model (original emoji, folded emoticon, name text,
> sentiment tag) so each consumer picks the layer it needs. Wire the
> annotations as optional input to Sentiment Detection, the Name Finder (name
> text plus entity-type as a gazetteer-like signal), and the Document
> Categorizer. Each annotation dimension is opt-in, so a consumer that only
> wants the folded text pays nothing for the rest.
> *Acceptance criteria.*
> * Optional annotation dimensions (name, sentiment, entity type, category),
> each with its own provenance/source field, surfaced through {{Term}}.
> * Wired as optional features into Sentiment Detection, Name Finder, and
> Document Categorizer.
> * Each dimension independently opt-in; default behavior unchanged.
> * Licensing cleared and documented for every bundled annotation source (see
> epic-level LEGAL item).
> h2. Out of scope (for the whole epic)
> * DL whitespace/dash offset-safety (done in 1850:
> {{NameFinderDL.findInOriginal}}, {{AbstractDL.normalizeInputAligned}}).
> * The {{Alignment}} model and {{andThen}} themselves (delivered in 1850; this
> epic consumes them).
> * Changing any default normalization behavior. This epic adds offset-aware
> capability and the provenance/annotation model; it does not turn expanding
> folds on by default.
> h2. Open questions to close in design review
> # The gate (NFC/NFKC/case-fold edits): confirm option (c) for the MVP, or
> take ICU4J. Gates OPENNLP-1867.
> # CSV vs annotated {{.txt}} for the lookups; one combined file or one per
> fold type.
> # Offset-aware method on {{CharSequenceNormalizer}} directly vs a parallel
> capability interface.
> # Default aggressiveness: full case folding and NFKC change tokens visibly,
> so likely opt-in, not default.
> # Whether {{UNSPECIFIED}} entries are allowed in a release build at all, or
> each must be reviewed and signed off.
> # Which dataset anchors each annotation dimension (emoji-sentiment ranking,
> CLDR for names, and what for entity type and category); same file or a
> separate keyed file.
> # Emoji/emoticon as punctuation vs a dedicated {{WordType}} for counting and
> feature generation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
