This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch sentencepiece
in repository https://gitbox.apache.org/repos/asf/opennlp.git
discard 6c19162e8 OPENNLP-1885: Guard tokenizer deserialization with an
allow-listing ObjectInputFilter
omit 7f20e7663 OPENNLP-1885: Make the tokenizer graph serializable with
computed UIDs, name the format constants, document every helper
omit 44177ce17 OPENNLP-1885: Document subword tokenization in the manual
omit 161e799cc OPENNLP-1885: Trim residual commentary per review conventions
omit b87d457ae OPENNLP-1885: Declare serialVersionUID on
SentencePieceTokenizer
omit 5a70734e1 OPENNLP-1885: Tighten javadoc to contracts and document
helpers and overrides
omit 029fa6c59 OPENNLP-1885: Trim commentary and tighten javadoc per review
conventions
omit d961a6509 OPENNLP-1885: Document the hand-rolled protobuf reader
rationale and refactor trigger
omit 3ace75218 OPENNLP-1885: Add WordpieceEncoder and fold the unreleased
BertTokenizer into it
omit 01863a8a5 OPENNLP-1885: Move the subword contract into opennlp-api
omit 3fb8b6bf5 Speed up the encode path 2.3x, parity-checked at every step
omit 4638249b8 Add opennlp-subword: pure-Java SentencePiece inference with
exact original-text spans
add e797262e8 OPENNLP-1885: Add opennlp-subword: pure-Java SentencePiece
inference with exact original-text spans
add cb707afa1 OPENNLP-1885: Speed up the encode path 2.3x, parity-checked
at every step
add 2ccdaf527 OPENNLP-1885: Move the subword contract into opennlp-api
add 3835e90b0 OPENNLP-1885: Add WordpieceEncoder and fold the unreleased
BertTokenizer into it
add 3c0a83902 OPENNLP-1885: Document the hand-rolled protobuf reader
rationale and refactor trigger
add 9b6ecd722 OPENNLP-1885: Trim commentary and tighten javadoc per review
conventions
add cdef6f20d OPENNLP-1885: Tighten javadoc to contracts and document
helpers and overrides
add 4c93b267c OPENNLP-1885: Declare serialVersionUID on
SentencePieceTokenizer
add 95ecac8e6 OPENNLP-1885: Trim residual commentary per review conventions
add f6464e6df OPENNLP-1885: Document subword tokenization in the manual
add 60a716b69 OPENNLP-1885: Make the tokenizer graph serializable with
computed UIDs, name the format constants, document every helper
add bcbb0cab5 OPENNLP-1885: Guard tokenizer deserialization with an
allow-listing ObjectInputFilter
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6c19162e8)
\
N -- N -- N refs/heads/sentencepiece (bcbb0cab5)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
