This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1850-4-docs
in repository https://gitbox.apache.org/repos/asf/opennlp.git
discard 0769f0cc6 OPENNLP-1850 Manual: qualify the dash-fold offset claim for
supplementary-plane dashes
discard 11b41bf4e OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
discard 6f23f79cc OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
discard 7a26d0af7 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
discard ee2d5a5b2 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
discard cb0c07346 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
discard bb05e8292 OPENNLP-1850 Tighten normalizer manual wording (review nits)
discard 8ab1e262d OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
discard fe5c740f9 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
discard 2d48b5ab7 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
discard 9fa0c8d87 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
discard 28cdca1ae OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
discard f3f545a2d OPENNLP-1850 DL review nits: document arg-validation
contract, javadoc locate(), hoist span-ordering constant
discard 5e00d4d8d OPENNLP-1850 DL hardening: IllegalArgumentException null
contract and null-element guards
discard 20520ec9c OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
discard 1e3ef2c06 OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
discard 29a83c891 OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
discard aee7377e4 OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
discard 73d3b2a54 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
discard 013254420 OPENNLP-1850 Fail loud on corrupt document-classification
model output
discard 39615df87 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
discard 094c1d4f9 OPENNLP-1850 Harden fail-loud paths in the DL components
discard 0165d1f27 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
discard 9d8f948a0 OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
discard 9542aa3d1 OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
discard 9512a32d7 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add d5d37dca2 OPENNLP-205: Refactor the end-of-sentence position to span
mapping in SentenceDetectorME (#1141)
add b42de7b68 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add c96cb1cca OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
add 7b326c45e OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
add 44f54e9b8 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
add b03e39224 OPENNLP-1850 Harden fail-loud paths in the DL components
add 22c3e1825 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
add b9abbcc03 OPENNLP-1850 Fail loud on corrupt document-classification
model output
add 45175b917 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
add 1aca3cb6e OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
add 81816d194 OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
add 8d810786c OPENNLP-1850 Review nits: extract testable DL guards;
merge-copy; capitalize msgs; migration note
add 5ed944991 OPENNLP-1850 Follow AlignedText.normalized() -> CharSequence
in NameFinderDL
add f260fe712 OPENNLP-1850 DL hardening: IllegalArgumentException null
contract and null-element guards
add e23658141 OPENNLP-1850 DL review nits: document arg-validation
contract, javadoc locate(), hoist span-ordering constant
add c8fc5f460 OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
add 2bd759ea8 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
add 7142a77f9 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
add 36cfa78b0 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
add 9f87cc1ec OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
add ddb15b47a OPENNLP-1850 Tighten normalizer manual wording (review nits)
add 6bd33e587 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
add cd8d9cd15 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
add 4fab66202 OPENNLP-1850 Docs review nits: declare xmlns:xlink; populate
second ids2Labels example
add edd419eba OPENNLP-1850 Review nits: manual uses
defaultChain()/matchingAnalyzer(); drop BM25/search framing
add 5c37b4278 OPENNLP-1850 Align normalizer.xml DOCTYPE to the
OPENNLP-1854 local DTD catalog form
add 6631a1368 OPENNLP-1850 Manual: qualify the dash-fold offset claim for
supplementary-plane dashes
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0769f0cc6)
\
N -- N -- N refs/heads/OPENNLP-1850-4-docs (6631a1368)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../tools/sentdetect/SentenceDetectorME.java | 161 ++++++-----
.../SentenceDetectorMESpanMappingTest.java | 308 +++++++++++++++++++++
2 files changed, 404 insertions(+), 65 deletions(-)
create mode 100644
opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/sentdetect/SentenceDetectorMESpanMappingTest.java
