Hi Lane, This is a phrase-based model built with the train-recaser.perl script. There is no reordering table, although the moses.ini file has these entries:
# distortion (reordering) weight [weight-d] 0.318856 and [distortion-limit] 6 Should I set the distortion limit to zero when there's no reordering table, or does Moses simply ignore it? Another interesting point, Moses continues running. Top reports Moses now consumes 20% of the memory. The system has 16GB RAM and it's configured with binarized ttable and lm. At idle, it consumes about 80 MB. I'll let it run until it finishes or crashes :) On 06/14/2013 10:56 PM, Lane Schwartz wrote: > Tom, > > Are you running phrase-based or hierarchical? If phrase-based, what is > your distortion limit? > > Lane > > On Fri, Jun 14, 2013 at 11:44 AM, Tom Hoar > <[email protected]> wrote: >> I build a custom recaser model that tokenizes the sentence into >> characters. It works well but it is slower than other models. In one >> batch, there is an unexpectedly long source sentence. Line 1017 (of 2400 >> lines) has 3,854 characters, which becomes 3,854 tokens as the Moses input. >> >> When it gets to this long line, all stdout pauses. Top shows Moses, >> configured to run on 6 cores, is running @ 598%. That's great. CPU >> processing continues at <600% CPU load for about 5-6 minutes. Then, it >> throttles back to "only" 100% load. >> >> Is is possible that when Moses drops to 100% load, it has finished >> processing the balance of the 2400 lines and the final thread continues >> processing that extremely long line? >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
