Hi, I’ve got a question on script tokenizer.perl. I’m wondering whether is it possible to get somewhere nonbreaking_prefix.* for various languages. Does exist such a place? Or, how I can tokenize a text file if I don’t have enough knowledge about the particular language.
Thanks, Tomas _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
