This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1868
in repository https://gitbox.apache.org/repos/asf/opennlp.git
discard 87f7bde6a OPENNLP-1868: Bundled full case-folding rung with provenance
and audit
discard 130b5ed48 OPENNLP-1850 Manual: qualify the dash-fold offset claim for
supplementary-plane dashes
discard 4f2fc0b28 OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
discard efbc88590 OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
discard 1b899aae6 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
discard 3a35c124b OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
discard c12d0f793 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
discard 1de3995b4 OPENNLP-1850 Tighten normalizer manual wording (review nits)
discard f3d8c6bb2 OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
discard ef552ef73 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
discard 3a229732c OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
discard cdd147300 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
discard f80b1b58b OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
discard 659f33ddc OPENNLP-1850 DL review nits: document arg-validation
contract, javadoc locate(), hoist span-ordering constant
discard 237932518 OPENNLP-1850 DL hardening: IllegalArgumentException null
contract and null-element guards
discard c33166f32 OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
discard b92141f0d OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
discard 2dd1783c8 OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
discard bcca50498 OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
discard d0f8a59b7 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
discard dc75d9e38 OPENNLP-1850 Fail loud on corrupt document-classification
model output
discard 6db5d1d1a OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
discard 4186a7b39 OPENNLP-1850 Harden fail-loud paths in the DL components
discard 61cf124da OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
discard f572435cb OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
discard 1f8e1f831 OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
discard 2ec0365cf OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
discard 19ae614df OPENNLP-1850 Review: validate NormalizationProfile components
discard cfc3de91d OPENNLP-1850 Review nits: rename
searchAnalyzer->matchingAnalyzer; drop 'search' framing in profile docs
discard c7934bd9a OPENNLP-1850 Review nits: add Turkish profile; derive
coverage from the enum (profiles)
discard cc10d1a8c OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
discard 827bf8e52 OPENNLP-1850 Per-language NormalizationProfile registry (2c)
discard fefa03c22 OPENNLP-1850 Review: thread-safe Term cache, argument
validation, drop 'canonical' framing
discard 37ce9e128 OPENNLP-1850 Term model hardening: fail loud on a
contract-violating Lemmatizer
discard 1cb0bb03a OPENNLP-1850 Review nits: TermAnalyzer javadoc references
matchingAnalyzer()
discard 69906de04 OPENNLP-1850 Review nits: rename dashes()->dash(); LEMMA
doc+test; soften forward-link (Term)
discard e67c8bd29 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
add 74092e1c9 OPENNLP-1873: Update ONNX runtime to 1.27.0 (#1143)
add 1712f32db OPENNLP-1874: Update Log4J to 2.26.1 (#1144)
add 945a3079f Minor: Regenerated NOTICE File for
1712f32dbe657792f9a80fa960018ab2a9eb2206 (#1147)
add 5a7b3aa6d OPENNLP-1863: Layered Term model — Term, TermAnalyzer (#1111)
add 784e9c85b OPENNLP-1864: Per-language NormalizationProfile registry
(2c/7) (#1149)
add d5d37dca2 OPENNLP-205: Refactor the end-of-sentence position to span
mapping in SentenceDetectorME (#1141)
add b42de7b68 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add c96cb1cca OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
add 7b326c45e OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
add 44f54e9b8 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
add b03e39224 OPENNLP-1850 Harden fail-loud paths in the DL components
add 22c3e1825 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
add b9abbcc03 OPENNLP-1850 Fail loud on corrupt document-classification
model output
add 45175b917 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
add 1aca3cb6e OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
add 81816d194 OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
add 8d810786c OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
add 5ed944991 OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
add f260fe712 OPENNLP-1850 DL hardening: IllegalArgumentException null
contract and null-element guards
add e23658141 OPENNLP-1850 DL review nits: document arg-validation
contract, javadoc locate(), hoist span-ordering constant
add c8fc5f460 OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
add 2bd759ea8 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
add 7142a77f9 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
add 36cfa78b0 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
add 9f87cc1ec OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
add ddb15b47a OPENNLP-1850 Tighten normalizer manual wording (review nits)
add 6bd33e587 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
add cd8d9cd15 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
add 4fab66202 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
add edd419eba OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
add 5c37b4278 OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
add 6631a1368 OPENNLP-1850 Manual: qualify the dash-fold offset claim for
supplementary-plane dashes
add 0ef9c9392 OPENNLP-1868: Bundled full case-folding rung with provenance
and audit
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (87f7bde6a)
\
N -- N -- N refs/heads/OPENNLP-1868 (0ef9c9392)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
NOTICE | 10 +-
.../tools/sentdetect/SentenceDetectorME.java | 161 ++++++-----
.../util/normalizer/NormalizationProfile.java | 14 +-
.../util/normalizer/NormalizationProfiles.java | 26 +-
.../java/opennlp/tools/util/normalizer/Term.java | 18 +-
.../tools/util/normalizer/TermAnalyzer.java | 77 ++++--
.../SentenceDetectorMESpanMappingTest.java | 308 +++++++++++++++++++++
.../util/normalizer/NormalizationProfilesTest.java | 28 +-
.../normalizer/TermAnalyzerMultilingualTest.java | 78 ++++++
.../tools/util/normalizer/TermAnalyzerTest.java | 26 +-
.../opennlp/tools/util/normalizer/TermTest.java | 109 ++++++++
opennlp-distr/src/main/readme/NOTICE | 10 +-
pom.xml | 4 +-
13 files changed, 729 insertions(+), 140 deletions(-)
create mode 100644
opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/sentdetect/SentenceDetectorMESpanMappingTest.java
create mode 100644
opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/util/normalizer/TermAnalyzerMultilingualTest.java
create mode 100644
opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/util/normalizer/TermTest.java
