it's difficult to tell but I would say the mgiza executables isn't the problem. It's probably to do with running out of disk space or memory.
the snt2coooc executable in mgiza uses a lot of memory so may have been killed by the OS. The phrase table creation requires a lot of disk space to sort intermediate files. I would monitor those 2 things Hieu Hoang http://statmt.org/hieu On 31 July 2018 at 20:41, James Baker <[email protected]> wrote: > Hi, > > I'm having some peculiar issues with MGiza++. Using MGiza and Moses, I've > successfully built some translation models on my Ubuntu 16.04 desktop > machine. I'd now like to do the same thing, but on a machine hosted in AWS. > > I'm using the same operating system, and as far as I can tell all my > versions are identical. The build of MGiza++ runs fine, reports no errors, > and produces output the same as on my desktop machine. However, when I try > to build the models, I get a whole load of errors and the resultant models > are empty (64 bytes for the reordering model, 0 bytes for the translation > model - the language model builds fine). > > The first "errors" I can see in the log seem to occur on stage 4 of the > Moses training script (train-model.perl): > > (4) generate lexical translation table 0-0 @ Tue Jul 31 10:22:58 UTC > 2018 > (/opt/model-builder/training/data.ru,/opt/model-builder/ > training/data.en,/opt/model-builder/training/model/lex) > !Argument "anna" isn't numeric in numeric ge (>=) at /opt/model-builder/ > mosesdecoder/scripts/training/LexicalTranslationModel.pm line 112, <A> > line 1. > Use of uninitialized value $ei in numeric ge (>=) at /opt/model-builder/ > mosesdecoder/scripts/training/LexicalTranslationModel.pm line 112, <A> > line 1. > Use of uninitialized value $ei in hash element at /opt/model-builder/ > mosesdecoder/scripts/training/LexicalTranslationModel.pm line 118, <A> > line 1. > Use of uninitialized value $ei in array element at /opt/model-builder/ > mosesdecoder/scripts/training/LexicalTranslationModel.pm line 121, <A> > line 1. > Use of uninitialized value $ei in array element at /opt/model-builder/ > mosesdecoder/scripts/training/LexicalTranslationModel.pm line 123, <A> > line 1. > ... > > There are a large number of errors of that nature, and following those > errors there are additional errors but I suspect these are caused by the > fact that this stage is failing. > > It's possible that there are earlier problems, but I'm not really sure > what to be looking for in the logs (for instance - there are some lines > warning about alignments in Model2 being 0 - is that an issue?). > > If I replace the MGiza binaries built on the AWS machine with the binaries > built on my desktop, it runs fine - so I know it's an issue with MGiza and > presumably something to do with my build. The commands I'm running to build > and install are as follows > > git clone https://github.com/moses-smt/mgiza.git > cd mgiza/mgizapp > cmake . > make > make install > cp bin/* ../../mosesdecoder/bin > cp scripts/merge_alignment.py ../../mosesdecoder/bin > > As I mentioned previously, these commands work fine on my desktop machine > which should be a very similar (if not identical) set up. > > Does anyone have any ideas as to what might be causing the problem (or, > more importantly, what I can do to fix it)? > > Thanks in advance, > James > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
