Thanks Hieu, I'll give that a go this morning and keep an eye on the disk space and RAM, although I would be surprised if that was the problem (I've got <3GB of training data, 64GB of RAM, and 100GB of disk space). It also wouldn't explain why binaries built on a different machine work, but binaries built on the same machine don't.
Any other ideas for things I should be checking? Cheers, James On Wed, 1 Aug 2018 at 03:03, Hieu Hoang <[email protected]> wrote: > it's difficult to tell but I would say the mgiza executables isn't the > problem. It's probably to do with running out of disk space or memory. > > the snt2coooc executable in mgiza uses a lot of memory so may have been > killed by the OS. The phrase table creation requires a lot of disk space to > sort intermediate files. > > I would monitor those 2 things > > Hieu Hoang > http://statmt.org/hieu > > On 31 July 2018 at 20:41, James Baker <[email protected]> wrote: > >> Hi, >> >> I'm having some peculiar issues with MGiza++. Using MGiza and Moses, I've >> successfully built some translation models on my Ubuntu 16.04 desktop >> machine. I'd now like to do the same thing, but on a machine hosted in AWS. >> >> I'm using the same operating system, and as far as I can tell all my >> versions are identical. The build of MGiza++ runs fine, reports no errors, >> and produces output the same as on my desktop machine. However, when I try >> to build the models, I get a whole load of errors and the resultant models >> are empty (64 bytes for the reordering model, 0 bytes for the translation >> model - the language model builds fine). >> >> The first "errors" I can see in the log seem to occur on stage 4 of the >> Moses training script (train-model.perl): >> >> (4) generate lexical translation table 0-0 @ Tue Jul 31 10:22:58 UTC >> 2018 >> (/opt/model-builder/training/data.ru >> ,/opt/model-builder/training/data.en,/opt/model-builder/training/model/lex) >> !Argument "anna" isn't numeric in numeric ge (>=) at >> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm >> line 112, <A> line 1. >> Use of uninitialized value $ei in numeric ge (>=) at >> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm >> line 112, <A> line 1. >> Use of uninitialized value $ei in hash element at >> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm >> line 118, <A> line 1. >> Use of uninitialized value $ei in array element at >> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm >> line 121, <A> line 1. >> Use of uninitialized value $ei in array element at >> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm >> line 123, <A> line 1. >> ... >> >> There are a large number of errors of that nature, and following those >> errors there are additional errors but I suspect these are caused by the >> fact that this stage is failing. >> >> It's possible that there are earlier problems, but I'm not really sure >> what to be looking for in the logs (for instance - there are some lines >> warning about alignments in Model2 being 0 - is that an issue?). >> >> If I replace the MGiza binaries built on the AWS machine with the >> binaries built on my desktop, it runs fine - so I know it's an issue with >> MGiza and presumably something to do with my build. The commands I'm >> running to build and install are as follows >> >> git clone https://github.com/moses-smt/mgiza.git >> cd mgiza/mgizapp >> cmake . >> make >> make install >> cp bin/* ../../mosesdecoder/bin >> cp scripts/merge_alignment.py ../../mosesdecoder/bin >> >> As I mentioned previously, these commands work fine on my desktop machine >> which should be a very similar (if not identical) set up. >> >> Does anyone have any ideas as to what might be causing the problem (or, >> more importantly, what I can do to fix it)? >> >> Thanks in advance, >> James >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
