This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1894-lattice-cjk
in repository https://gitbox.apache.org/repos/asf/opennlp.git
from b5723b45d OPENNLP-1894: Apply the review-convention pass and drop
unreferenced lexicon accessors
add 65579e9cd OPENNLP-1894: Trim parsed lines as Unicode whitespace and
document the tokenizer overrides
No new revisions were added by this update.
Summary of changes:
.../opennlp/tools/tokenize/lattice/LatticeTokenizer.java | 16 +++++++++++++++-
.../opennlp/tools/tokenize/lattice/MecabDictionary.java | 8 ++++----
.../opennlp/tools/tokenize/lattice/UnigramSegmenter.java | 16 +++++++++++++++-
.../tools/tokenize/lattice/UnigramSegmenterTest.java | 15 +++++++++++++++
4 files changed, 49 insertions(+), 6 deletions(-)
