Souhir, I don't know if it's "normal" but it is not uncommon. There are many variables that contribute to the training time and it's difficult to predict how long it will take. Is your host is the same as when you trained the larger phrase table? Are you using the same configuration options? If so, and the data is the only thing that changed, there are several reasons the training can take longer.
6.5M phrase pairs is only one measure of the parallel corpus. Total token count, average/max tokens per pair, complexity of patterns within the phrases and between the pairs, repetitions of phrase segments (n-grams), etc. also contribute to the training time. Accounting for all of these variables (and more), can cause a smaller parallel corpus to consume more memory than a larger corpus. "Running very slowly and not running constantly" could be signs your system is visualizing memory to the swap partition. Have you checked your host's available memory? Have you run out of RAM and are relying on the system swap partition? On Thu, 5 Jul 2012 16:37:06 +0200, Souhir Gahbiche <[email protected]> wrote: > Hi all, > > I'm scoring a huge phrase-table (6.5M phrase pairs). Moses is running > since 4 days and stills in step 6.2. > The program seems running very slowly and not running constantly. > I remember, I trained a huge phrase table bigger than this but the > training was faster. > Is it normal that training stills all this time? > > Regards > -S > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
