Uli, I sent the phrase-table to Marcin yesterday to test - He was able to binarize the table successfully. Here, we’ve been compiling moses with the Intel compiler. We built the same checkout with gcc and using processPhraseTableMin from that build we were able to successfully binarize the phrase table.
One thing I saw during testing these different configs was the intel-compiled version would output tcmalloc debug messages, but the gcc-compiled one would not. We’re using tcmalloc-minimal for these builds. Should we be using the full version? Running moses —version on both builds shows Boost 1.54, Xmlrpc-c 1.33.17 and CMPH (version unknown) linked in. We compile static binaries on a RHEL 6-based distro (Scientific Linux 6.7) -Jeremy > Message: 2 > Date: Thu, 4 Feb 2016 15:03:02 +0000 > From: Ulrich Germann <[email protected]> > Subject: Re: [Moses-support] Problem with processPhraseTableMin > To: Marcin Junczys-Dowmunt <[email protected]> > Cc: "[email protected]" <[email protected]> > Message-ID: > <cahqsruq_gtrcubkzwmzpvmkypormygse4sw-4rybs_jzml1...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > I've had processPhraseTableMin crash when the phrase table contains > duplicate entries (can't remember if there was an unreasonable memory > allocation involved). Is Marcin using the exact same phrase table? Can you > check if the phrase table has duplicate entries? > > To crash or not to crash could also depend on OS and libraries used. You > can get the versions of libraries compiled into moses with > > moses --version > > I've had duplicate entries in the phrase table after running > ptable-sigtest-filter, which is Marcin's implementation of Johnson et al.'s > significance filtering that I pulled in from his WIPO branch; compile with > --with-mm --with-mm-extras to get it compiled. > > - Uli _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
