Hi Amir You can use this tokeniser as a basis for creating your own tokeniser, or you can swap in your own tokeniser. For EMS a tokeniser should read from stdin and write to stdout, so you can run it like this
tokeniser [options] < input > output cheers - Barry On 24/01/14 08:58, amir haghighi wrote: > I use the built-in tokenizer in the Moses. > how can I change this tokenizer? should I change the source code? > > Regards > Amir > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
