Hi, this may be also caused by having too long / empty / length-mismatched sentences when running GIZA. Make sure to run the clean-corpus-n.perl script first.
-phi On Thu, Oct 9, 2014 at 10:49 AM, Matthias Huck <[email protected]> wrote: > Hi Arefeh, > > Have you been able to resolve that issue? Maybe one of your GIZA > alignments is flawed, for instance because the GIZA process was > terminated before is finished. Did you check that both the standard and > the inverse alignment files have the same number of lines? > > Check it like this: > > $ zcat training/giza.1/de-en.A3.final.gz | wc -l; zcat > training/giza-inverse.1/en-de.A3.final.gz | wc -l > 900000 > 501713 > > In that case there would be a problem and you'd have to rerun GIZA in > the inverse direction. If you get the same number of lines and it > matches what you expect to get from your corpus, then it's a different > issue and you have to investigate further. > > Cheers, > Matthias > > > On Mon, 2014-10-06 at 03:13 -0700, Arefeh Kazemi wrote: > > Hi > > I have re-installed moses on my system but I have a problem with giza > > - symmetrize step. > > it gets some errors of this type: > > Sentence mismatch error! Line #501714 > > Sentence mismatch error! Line #501715 > > . > > . > > . > > Sentence mismatch error! Line #900000 > > > > > > all of my data files are in utf8 format and I have run moses > > successfully on these files before. > > > > > > any suggestion to fix the problem would be appreciated. > > > > > > Regards > > Arefeh > > > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
