I didn't realize that there was a compound splitting script that was distributed with Moses, but I suspect it will be quite a bit different. The one in cdec is based on CRFs and uses a bunch of different features to model segmentations, and it will also produce segmentation lattices. It was trained using segmentations that seemed (to my intuition) to be "sensible" for MT.
On Fri, Mar 11, 2011 at 3:05 AM, Joerg Tiedemann <jorg.tiedem...@lingfil.uu.se> wrote: > On Thu, Mar 10, 2011 at 8:09 PM, Chris Dyer <cd...@cs.cmu.edu> wrote: >> There's a German compound splitting tool that's tuned for MT that's >> released as part of cdec (https://github.com/redpony/cdec). You'll >> have to build the decoder, but then you should be able to run the >> script in >> >> cdec / compound-split / compound-split.pl > > Does this use the same idea as the compound-splitter script > distributed with Moses? > Are there any known performance differences? > > Jörg > > >> >> -Chris >> >> On Thu, Mar 10, 2011 at 1:50 PM, Tom Hoar >> <tah...@precisiontranslationtools.com> wrote: >>> I know German language requires special corpus preparation. Can someone >>> point me in the right direction regarding what compound words, stemming, >>> etc? >>> >>> Thanks, >>> Tom >>> >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > > -- > ********************************************************************************** > Jörg > Tiedemann jorg.tiedem...@lingfil.uu.se > Dep. of Linguistics and Philology > http://stp.lingfil.uu.se/~joerg/ > Uppsala University tel: +46 (0)18 - 471 > 1412 > Box 635, SE-751 26 Uppsala/SWEDEN fax: +46 (0)18 - 471 1094 > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
