Ok, this is weird. We just ran the intel+tcmalloc debug variant and it completed as normal. Also noticed no tcmalloc debug messages though.
> On Feb 4, 2016, at 12:01 PM, [email protected] wrote: > > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. Re: Problem with processPhraseTableMin (Kenneth Heafield) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 4 Feb 2016 15:51:12 +0000 > From: Kenneth Heafield <[email protected]> > Subject: Re: [Moses-support] Problem with processPhraseTableMin > To: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset=UTF-8 > > I can haz backtrace? It's clearly calculating a huge amount to allocate > somewhere which is leading to tcmalloc returning NULL but that's not > tcmalloc's fault. > > On 02/04/2016 03:47 PM, Marcin Junczys-Dowmunt wrote: >> I've been using it with and without tcmalloc with no problems, the part >> where it crashes for is not multi-threading anyway. I guess it's the >> intel compiler, no idea why though. >> >> W dniu 2016-02-04 16:37, Jeremy Gwinnup napisa?(a): >> >>> Uli, >>> >>> I sent the phrase-table to Marcin yesterday to test - He was able to >>> binarize the table successfully. Here, we've been compiling moses with the >>> Intel compiler. We built the same checkout with gcc and using >>> processPhraseTableMin from that build we were able to successfully binarize >>> the phrase table. >>> >>> One thing I saw during testing these different configs was the >>> intel-compiled version would output tcmalloc debug messages, but the >>> gcc-compiled one would not. We're using tcmalloc-minimal for these builds. >>> Should we be using the full version? >>> >>> Running moses ?version on both builds shows Boost 1.54, Xmlrpc-c 1.33.17 >>> and CMPH (version unknown) linked in. We compile static binaries on a RHEL >>> 6-based distro (Scientific Linux 6.7) >>> >>> -Jeremy >>>> Message: 2 Date: Thu, 4 Feb 2016 15:03:02 +0000 From: Ulrich Germann >>>> <[email protected] <mailto:[email protected]>> Subject: >>>> Re: [Moses-support] Problem with processPhraseTableMin To: Marcin >>>> Junczys-Dowmunt <[email protected] <mailto:[email protected]>> Cc: >>>> "[email protected] <mailto:[email protected]>" >>>> <[email protected] <mailto:[email protected]>> Message-ID: >>>> <cahqsruq_gtrcubkzwmzpvmkypormygse4sw-4rybs_jzml1...@mail.gmail.com >>>> <mailto:cahqsruq_gtrcubkzwmzpvmkypormygse4sw-4rybs_jzml1...@mail.gmail.com>> >>>> Content-Type: text/plain; charset="utf-8" I've had >>>> processPhraseTableMin crash when the phrase table contains duplicate >>>> entries (can't remember if there was an unreasonable memory >>>> allocation involved). Is Marcin using the exact same phrase table? >>>> Can you check if the phrase table has duplicate entries? To crash or >>>> not to crash could also depend on OS and libraries used. You can get >>>> the versions of libraries compiled into moses with moses --version >>>> I've had duplicate entries in the phrase table after running >>>> ptable-sigtest-filter, which is Marcin's implementation of Johnson et >>>> al.'s significance filtering that I pulled in from his WIPO branch; >>>> compile with --with-mm --with-mm-extras to get it compiled. - Uli >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] <mailto:[email protected]> >>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 112, Issue 14 > ********************************************** _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
