We’ve had reasonable luck with the Stanford Chinese segmenter - I think the ctb model did better than the pku one for our use case
> Message: 2 > Date: Fri, 20 Mar 2015 13:19:02 +0100 > From: Marcin Junczys-Dowmunt <[email protected]> > Subject: [Moses-support] Chinese segmentation/tokenization > To: Moses Support <[email protected]> > Message-ID: <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > > > Hi, > > questions appear from time to time on the list concerning Chinese > segmentation/tokenization. I saw Barry mention Lingpipe and other tools. > Is there a favourite tool you guys prefer to use over others? > > Thanks, > > Marcin _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
