I can haz backtrace? It's clearly calculating a huge amount to allocate somewhere which is leading to tcmalloc returning NULL but that's not tcmalloc's fault.
On 02/04/2016 03:47 PM, Marcin Junczys-Dowmunt wrote: > I've been using it with and without tcmalloc with no problems, the part > where it crashes for is not multi-threading anyway. I guess it's the > intel compiler, no idea why though. > > W dniu 2016-02-04 16:37, Jeremy Gwinnup napisał(a): > >> Uli, >> >> I sent the phrase-table to Marcin yesterday to test - He was able to >> binarize the table successfully. Here, we've been compiling moses with the >> Intel compiler. We built the same checkout with gcc and using >> processPhraseTableMin from that build we were able to successfully binarize >> the phrase table. >> >> One thing I saw during testing these different configs was the >> intel-compiled version would output tcmalloc debug messages, but the >> gcc-compiled one would not. We're using tcmalloc-minimal for these builds. >> Should we be using the full version? >> >> Running moses —version on both builds shows Boost 1.54, Xmlrpc-c 1.33.17 >> and CMPH (version unknown) linked in. We compile static binaries on a RHEL >> 6-based distro (Scientific Linux 6.7) >> >> -Jeremy >>> Message: 2 Date: Thu, 4 Feb 2016 15:03:02 +0000 From: Ulrich Germann >>> <[email protected] <mailto:[email protected]>> Subject: >>> Re: [Moses-support] Problem with processPhraseTableMin To: Marcin >>> Junczys-Dowmunt <[email protected] <mailto:[email protected]>> Cc: >>> "[email protected] <mailto:[email protected]>" >>> <[email protected] <mailto:[email protected]>> Message-ID: >>> <cahqsruq_gtrcubkzwmzpvmkypormygse4sw-4rybs_jzml1...@mail.gmail.com >>> <mailto:cahqsruq_gtrcubkzwmzpvmkypormygse4sw-4rybs_jzml1...@mail.gmail.com>> >>> Content-Type: text/plain; charset="utf-8" I've had >>> processPhraseTableMin crash when the phrase table contains duplicate >>> entries (can't remember if there was an unreasonable memory >>> allocation involved). Is Marcin using the exact same phrase table? >>> Can you check if the phrase table has duplicate entries? To crash or >>> not to crash could also depend on OS and libraries used. You can get >>> the versions of libraries compiled into moses with moses --version >>> I've had duplicate entries in the phrase table after running >>> ptable-sigtest-filter, which is Marcin's implementation of Johnson et >>> al.'s significance filtering that I pulled in from his WIPO branch; >>> compile with --with-mm --with-mm-extras to get it compiled. - Uli >> _______________________________________________ >> Moses-support mailing list >> [email protected] <mailto:[email protected]> >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
