This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1894-lattice-cjk
in repository https://gitbox.apache.org/repos/asf/opennlp.git
at b5723b45d OPENNLP-1894: Apply the review-convention pass and drop
unreferenced lexicon accessors
This branch includes the following new commits:
new 3ce137626 OPENNLP-1894: Lattice segmentation over user-supplied
mecab-format dictionaries
new 1132c127d OPENNLP-1894: Character trie for lattice prefix search
new e29862206 OPENNLP-1894: Frequency-driven segmentation over
user-supplied lexicons
new 8730659ac OPENNLP-1894: Usage example and edge-case tests for the
lattice and unigram segmenters, corrected javadoc
new 040ea88dd OPENNLP-1894: Keep lattice test sources ASCII-only via
Unicode escapes, add an EUC-JP loading example
new 759af0e64 OPENNLP-1894: Document dictionary acquisition with a
checksum-verifying download helper
new 4d76e948d OPENNLP-1894: Categorize by code point, keep unknown
candidates inside their category run, and validate context ids at load
new b48a8eb38 OPENNLP-1894: Precompute category runs, unbox the trie, and
reject inexpressible dictionary values
new ae38eb423 OPENNLP-1894: Hold the lexicon in a double-array trie with
frequency-recoded labels
new 73d8aa53d OPENNLP-1894: Chain lattice nodes intrusively instead of
allocating per-position lists
new 104cd0e29 OPENNLP-1894: Document lattice CJK tokenization with a
mirror-tested example
new b5723b45d OPENNLP-1894: Apply the review-convention pass and drop
unreferenced lexicon accessors
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
