This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1850-4-docs
in repository https://gitbox.apache.org/repos/asf/opennlp.git
discard ed640258b OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
discard c51b2fec1 OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
discard 2625dfae9 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
discard aa8f60e89 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
discard 294192592 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
discard 50e725081 OPENNLP-1850 Tighten normalizer manual wording (review nits)
discard 5b7600f58 OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
discard 954c41f92 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
discard 2b3e39e80 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
discard bdd8ce102 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
discard 21fe81c47 OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
discard 43128bf8d OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
discard 95f631378 OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
discard 8170de6fb OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
discard 1fb2dd40c OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
discard 45a105240 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
discard 56f389912 OPENNLP-1850 Fail loud on corrupt document-classification
model output
discard dfc178a25 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
discard 1c2ebedff OPENNLP-1850 Harden fail-loud paths in the DL components
discard 983dff35b OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
discard bfd4b24ab OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
discard 1fed9a14a OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
discard 683fbda7c OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
discard 243f21edf OPENNLP-1850 Review nits: rename
searchAnalyzer->matchingAnalyzer; drop 'search' framing in profile docs
discard eb2692f16 OPENNLP-1850 Review nits: add Turkish profile; derive
coverage from the enum (profiles)
discard 6a446d1e0 OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
discard 8eb056c3c OPENNLP-1850 Per-language NormalizationProfile registry (2c)
discard 4842b1cad OPENNLP-1850 Review nits: TermAnalyzer javadoc references
matchingAnalyzer()
discard 90aa1d014 OPENNLP-1850 Review nits: rename dashes()->dash(); LEMMA
doc+test; soften forward-link (Term)
discard 6f1f7ceb4 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
add 4dbd6a2e9 OPENNLP-1850 Perf: hoist the per-char volatile reads in
WordBreakProperty/ExtendedPictographic
add dfb687ac8 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
add 1ccc77263 OPENNLP-1850 Review nits: rename dashes()->dash(); LEMMA
doc+test; soften forward-link (Term)
add b0b2fa653 OPENNLP-1850 Review nits: TermAnalyzer javadoc references
matchingAnalyzer()
add 11efd78c7 OPENNLP-1850 Per-language NormalizationProfile registry (2c)
add b7e574912 OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
add f18960131 OPENNLP-1850 Review nits: add Turkish profile; derive
coverage from the enum (profiles)
add ea80f2f83 OPENNLP-1850 Review nits: rename
searchAnalyzer->matchingAnalyzer; drop 'search' framing in profile docs
add 1833b088b OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add 1bcad3c9a OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
add e703f024e OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
add daba0a5a7 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
add 4455f866c OPENNLP-1850 Harden fail-loud paths in the DL components
add 6b0283cd5 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
add 6464d7319 OPENNLP-1850 Fail loud on corrupt document-classification
model output
add 9d2e54d39 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
add 6701e527b OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
add f0ffd5f5e OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
add 2374e9c65 OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
add db3f5e812 OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
add 394a6ade1 OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
add 72be8aeb7 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
add 54e06dcb5 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
add 46d3d1dda OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
add 7fa90d5c3 OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
add b909d1723 OPENNLP-1850 Tighten normalizer manual wording (review nits)
add 090c5c96e OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
add 248f7af20 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
add c03c60030 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
add 6f2f456f1 OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
add ca3e7e20a OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ed640258b)
\
N -- N -- N refs/heads/OPENNLP-1850-4-docs (ca3e7e20a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../tools/tokenize/uax29/ExtendedPictographic.java | 13 +++++++--
.../tools/tokenize/uax29/WordBreakProperty.java | 32 +++++++++++++++-----
.../tools/tokenize/uax29/WordSegmenter.java | 34 +++++++++++++++-------
.../opennlp/tools/tokenize/uax29/WordType.java | 7 ++++-
.../tokenize/uax29/ExtendedPictographicTest.java | 12 ++++++++
.../tokenize/uax29/WordBreakPropertyTest.java | 12 ++++++++
6 files changed, 88 insertions(+), 22 deletions(-)
