Kristian Rickert created OPENNLP-1868:
-----------------------------------------
Summary: Bundled full case-folding rung with provenance and audit
Key: OPENNLP-1868
URL: https://issues.apache.org/jira/browse/OPENNLP-1868
Project: OpenNLP
Issue Type: Task
Components: dl, Sentiment Detection
Affects Versions: 3.0.0-M4
Reporter: Kristian Rickert
Assignee: Kristian Rickert
Part 2 of 4 for the Epic OPENNLP-1852
*Stacks on:* OPENNLP-1867.
*Summary.* Add a bundled case-folding table sourced from Unicode
\{{CaseFolding.txt}} (status C common and F full, including the expanding folds
such as eszett to ss and ligature ff to ff), exposed as an offset-aware fold
rung. Because the table is authored with known source and target lengths, the
rung builds its \{{Alignment}} during the pass, so full case folding is
offset-safe without ICU4J. Carry provenance metadata per row
(\{{UTS21:CaseFolding:<status>}}).
*Acceptance criteria.*
* An offset-aware full-case-fold rung backed by the bundled table, opt-in (not
a default).
* A build/test completeness audit that fails on drift or gaps versus upstream
\{{CaseFolding.txt}} (every C and F row present and identical).
* Round-trip tests for representative expanding folds (eszett, ligatures)
through the pipeline.
* Bundled data carries LICENSE/NOTICE/rat-excludes, same discipline as the 1850
Unicode data files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
