Er, no. As far as I can observe, --parallel parallelizes some tasks that are possible into 2 threads, one for e2f, one for f2e. For step 2, giza++, it starts two threads, one for e2f and one for f2e. Then _that_ is parallelized by mgiza into whatever number of processes I tell it to (the original idea for mgiza is one process per CPU/CPU core). I beleive the solution would be not to parallelize step 2 from train-model.perl but leave it to mgiza if we're training with that. (Until then the workaround is to halve the number of cpus in the mgiza parameters I guess.)
I have yet to (and will soon) test the performance differences between 16 processes on a 8-core system versus 8 processes on a 8-core system. In theory guess the latter should fare somewhat better. Best regards, Tamas 2011/11/30 Kenneth Heafield <mo...@kheafield.com>: > Wouldn't it run 64 processes? > > On 11/30/11 21:31, Kádár Tamás (KTamas) wrote: >> Hi >> >> With the option --parallel and running mgiza with whatever number of >> cpus, the training script parallelizes mgiza too... so if I run >> --mgiza --mgiza-cpus 8 --parallel on an 8-core machine, I actually get >> 16 giza processes. Shouldn't it either a) warn you about this behavior >> or b) not parallelize mgiza? Makes no sense to me to parallelize >> something that is already parallelized :) >> >> Just my $.02. >> >> Best regards, >> Tamas >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
