Hi Tamas The behaviour seems reasonable to me.
The flag --parallel tells train-model.perl to run processes simultaneously if possible, so it will run giza in both directions at the same time. The --mgiza-cpus is just for mgiza. Why not just specify 4 cpus for mgiza, if you only have 8 in total? > >> --mgiza --mgiza-cpus 8 --parallel on an 8-core machine, I actually get > >> 16 giza processes. That's odd, I'd expect 2 giza processes, each with 8 threads. cheers - Barry On Wednesday 30 Nov 2011 22:05:40 Kádár Tamás (KTamas) wrote: > Er, no. As far as I can observe, --parallel parallelizes some tasks > that are possible into 2 threads, one for e2f, one for f2e. For step > 2, giza++, it starts two threads, one for e2f and one for f2e. Then > _that_ is parallelized by mgiza into whatever number of processes I > tell it to (the original idea for mgiza is one process per CPU/CPU > core). I beleive the solution would be not to parallelize step 2 from > train-model.perl but leave it to mgiza if we're training with that. > (Until then the workaround is to halve the number of cpus in the mgiza > parameters I guess.) > > I have yet to (and will soon) test the performance differences between > 16 processes on a 8-core system versus 8 processes on a 8-core system. > In theory guess the latter should fare somewhat better. > > Best regards, > Tamas > > 2011/11/30 Kenneth Heafield <mo...@kheafield.com>: > > Wouldn't it run 64 processes? > > > > On 11/30/11 21:31, Kádár Tamás (KTamas) wrote: > >> Hi > >> > >> With the option --parallel and running mgiza with whatever number of > >> cpus, the training script parallelizes mgiza too... so if I run > >> --mgiza --mgiza-cpus 8 --parallel on an 8-core machine, I actually get > >> 16 giza processes. Shouldn't it either a) warn you about this behavior > >> or b) not parallelize mgiza? Makes no sense to me to parallelize > >> something that is already parallelized :) > >> > >> Just my $.02. > >> > >> Best regards, > >> Tamas > >> _______________________________________________ > >> Moses-support mailing list > >> Moses-support@mit.edu > >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
