Hi guys, In the process of decoding, I usually tokenise material first, then decoding, at last detokenising. But I found the results of detokenising were not satisfied, because some blank spaces generated in tokenising are not got rid of in detokenising. I'm wondering if I must tokenise material before decoding?
Thanks in advance. Best Regards, Henry _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
