Kristian Rickert created OPENNLP-1869:
-----------------------------------------
Summary: Offset-aware bidirectional emoji and emoticon rung
Key: OPENNLP-1869
URL: https://issues.apache.org/jira/browse/OPENNLP-1869
Project: OpenNLP
Issue Type: Task
Components: dl, Sentiment Detection
Affects Versions: 3.0.0-M4
Reporter: Kristian Rickert
Assignee: Kristian Rickert
*Stacks on:* OPENNLP-1868.
*Summary.* Add an opt-in, offset-aware rung that folds emoji to emoticon
(expansion) and emoticon to emoji (contraction), backed by a provenance-tagged
lookup of the same shape as OPENNLP-1868. A supplementary-plane emoji is also
the two-UTF-16-units-to-one case, so it exercises the contraction path. Most
rows are \{{UNSPECIFIED}} (no governing standard for the mapping); anchor on
CLDR where a direction can be (emoji short-name).
*Acceptance criteria.*
* A bidirectional, opt-in, offset-aware emoji/emoticon rung with round-trip
tests in both directions, including a supplementary-plane emoji.
* Lookup carries provenance (\{{EMOJI}}/\{{EMOTICON}} fold types,
\{{CLDR:annotation}} or \{{UNSPECIFIED}}).
* Consistent \{{WordType}}-level classification so emoji and emoticons are
treated as one class for token counts and ML feature generation (settle:
punctuation-like bucket vs a dedicated emoji \{{WordType}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
