Thanks! That's a good workaround!
On Thu, Dec 17, 2015 at 2:56 PM, Matthias Huck <[email protected]> wrote: > Hi, > > As an addendum: > > You can try a manual workaround. Run gunzip on extract.o.sorted.gz and > do lexical-reordering-score on the resulting plain text file. > > It might be inconvenient but would hopefully solve the issue. > > Cheers, > Matthias > > > On Thu, 2015-12-17 at 17:44 +0000, Matthias Huck wrote: >> Hi, >> >> It's a problem that apparently occurs very rarely, and as Guy mentioned, >> we were so far assuming that it's caused by a zlib bug. >> >> However, the zlib bug was (to my knowledge) fixed in zlib v1.2.8. >> This seems to be the bug fix: >> https://github.com/madler/zlib/commit/51370f365607fe14a6a7a1a27b3bd29d788f5e5b >> >> I've only encountered the issue once (and I'm training systems >> frequently). When I came across it, I executed the same command with a >> Moses compile on a different machine that was running an older version >> of OpenSuse, rather than Ubuntu 12.04. The problem did not exist on the >> old system. >> >> My guess is that it really is a zlib bug, but it would be worrying if >> switching to zlib v1.2.8 doesn't resolve it. >> >> Cheers, >> Matthias >> >> >> On Thu, 2015-12-17 at 09:18 -0800, Varden Wang wrote: >> > I seem to be having a very very similar issue. I have the exact same >> > lib package as Guy (but I upgraded from lib 1.2.3.4). I'm using the >> > commit SHA e211d752f6bc680094520482f190d0f805405c6c of the >> > mosesdecoder. The funny thing is that I trained on the very same setup >> > on different data sets without encountering this problem. >> > >> > My error: >> > >> > Executing: >> > /usr/local/google/home/varden/MOSES/mosesdecoder/scripts/../bin/lexical-reordering-score >> > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/extract.o.sorted.gz >> > 0.5 >> > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/reordering-table. >> > --model "wbe msd wbe-msd-bidirectional-fe" >> > >> > Lexical Reordering Scorer >> > >> > scores lexical reordering models of several types (hierarchical, >> > phrase-based and word-based-extraction >> > >> > terminate called after throwing an instance of 'util::GZException' >> > >> > what(): zlib encountered invalid distances set code -3 >> > >> > ERROR: Execution of: >> > /usr/local/google/home/varden/MOSES/mosesdecoder/scripts/../bin/lexical-reordering-score >> > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/extract.o.sorted.gz >> > 0.5 >> > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/reordering-table. >> > --model "wbe msd wbe-msd-bidirectional-fe" >> > >> > died with signal 6, with coredump >> > >> > Thanks, >> > >> > Varden >> > >> > On Mon, Dec 7, 2015 at 9:01 AM, <[email protected]> wrote: >> > > Send Moses-support mailing list submissions to >> > > [email protected] >> > > >> > > To subscribe or unsubscribe via the World Wide Web, visit >> > > http://mailman.mit.edu/mailman/listinfo/moses-support >> > > or, via email, send a message with subject or body 'help' to >> > > [email protected] >> > > >> > > You can reach the person managing the list at >> > > [email protected] >> > > >> > > When replying, please edit your Subject line so it is more specific >> > > than "Re: Contents of Moses-support digest..." >> > > >> > > >> > > Today's Topics: >> > > >> > > 1. Lexical reordering fails with zlib (Guy) >> > > >> > > >> > > ---------------------------------------------------------------------- >> > > >> > > Message: 1 >> > > Date: Mon, 7 Dec 2015 06:14:47 +0000 (UTC) >> > > From: Guy <[email protected]> >> > > Subject: [Moses-support] Lexical reordering fails with zlib >> > > To: [email protected] >> > > Message-ID: <[email protected]> >> > > Content-Type: text/plain; charset=us-ascii >> > > >> > > Hello everyone, >> > > >> > > I've just recently started to work with Moses and managed to build a >> > > couple >> > > of models without problems... until now. >> > > >> > > I was training a new system and I got this error when executing >> > > lexical-reordering-score: >> > > >> > > ../mosesdecoder/scripts/../bin/lexical-reordering-score >> > > /local/scratch/train/model/extract.o.sorted.gz 0.5 >> > > /local/scratch/train/model/reordering-table. --model "wbe msd >> > > wbe-msd-bidirectional-fe" >> > > Lexical Reordering Scorer >> > > scores lexical reordering models of several types (hierarchical, >> > > phrase-based and word-based-extraction >> > > terminate called after throwing an instance of 'util::GZException' >> > > what(): util/read_compressed.cc:163 in virtual std::size_t >> > > util::{anonymous}::GZip::Read(void*, std::size_t, util::ReadCompressed&) >> > > threw GZException'. >> > > zlib encountered invalid distances set code -3 >> > > Aborted >> > > >> > > I found an old post >> > > (http://permalink.gmane.org/gmane.comp.nlp.moses.user/10151) saying this >> > > was >> > > due to an apparent bug in zlib 1.2.3.4 on Ubuntu 12.04 and that >> > > upgrading to >> > > zlib 1.2.8 solves the problem. However, I already have zlib 1.2.8 (but on >> > > Ubuntu 14.04) and I still get this error. In case it helps, package name >> > > is >> > > zlib1g:amd64 version 1:1.2.8.dfsg-1ubuntu). >> > > >> > > It's a bit strange that I didn't stumble upon this problem when I trained >> > > previous systems. >> > > >> > > Any ideas on what to do? >> > > >> > > Thank you very much, >> > > Guy >> > > >> > > >> > > >> > > >> > > ------------------------------ >> > > >> > > _______________________________________________ >> > > Moses-support mailing list >> > > [email protected] >> > > http://mailman.mit.edu/mailman/listinfo/moses-support >> > > >> > > >> > > End of Moses-support Digest, Vol 110, Issue 16 >> > > ********************************************** >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> >> >> > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
