Steps to reproduce this error: $ ~/mosesdecoder.git/bin/lmplz -o 2 <<< "that is what happens ? cssd has > nothing more or voldemort or pastries in prague ." > === 1/5 Counting and sorting n-grams === > Reading /tmp/sh-thd-107574999377 (deleted) > > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > tcmalloc: large alloc 29442056192 bytes == 0x2ae2000 @ > tcmalloc: large alloc 78512136192 bytes == 0x6df1b4000 @ > > **************************************************************************************************** > Unigram tokens 16 types 18 > === 2/5 Calculating and sorting adjusted counts === > Chain sizes: 1:216 2:107979354931 > tcmalloc: large alloc 107979358208 bytes == 0x192b4b6000 @ > lmplz: ./util/fixed_array.hh:104: T& > util::FixedArray<T>::operator[](std::size_t) [with T = > lm::NGramStream<lm::builder::BuildingPayload>; std::size_t = long unsigned > int]: Assertion `i < size()' failed.
On Wed, Sep 30, 2015 at 11:41 AM, Kenneth Heafield <[email protected]> wrote: > That's bad. Would you mind sending me privately a minimal example of > the data that reproduces the problem? > > Kenneth > > On 09/30/2015 04:29 PM, Alex Martinez wrote: > > Hello, > > today I've pulled moses code and recompiled and some experiments (EMS) > > that were already working are failing on the LM training step with the > > following error: > > > > Executing: /opt/moses/bin/lmplz --text > > /home/alexmc/devel/toydata/process/lm/nc=pos.factored.1 --order 5 --arpa > > /home/alexmc/devel/toydata/process/lm/nc=pos.lm.1 --discount_fallback > > === 1/5 Counting and sorting n-grams === > > Reading /mnt/a62/devel/toydata/process/lm/nc=pos.factored.1 > > > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > > tcmalloc: large alloc 4753956864 bytes == 0x1f7c000 @ > > tcmalloc: large alloc 22185107456 bytes == 0x11d536000 @ > > > **************************************************************************************************** > > Unigram tokens 2433135 types 47 > > === 2/5 Calculating and sorting adjusted counts === > > Chain sizes: 1:564 2:2630656000 3:4932480000 4:7891967488 5:11509120000 > > tcmalloc: large alloc 11509121024 bytes == 0x1f7c000 @ > > tcmalloc: large alloc 2630656000 bytes == 0x2aff70000 @ > > tcmalloc: large alloc 4932485120 bytes == 0x34cc3a000 @ > > tcmalloc: large alloc 7891968000 bytes == 0x64933c000 @ > > lmplz: ./util/fixed_array.hh:104: T& > > util::FixedArray<T>::operator[](std::size_t) [with T = > > lm::NGramStream<lm::builder::BuildingPayload>; std::size_t = long > > unsigned int]: Assertion `i < size()' failed. > > > > I'm runing a Linux server with Ubuntu 15.04 > > > > Any help will be appreciated > > > > Alex MartÃnez > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- When a place gets crowded enough to require ID's, social collapse is not far away. It is time to go elsewhere. The best thing about space travel is that it made it possible to go elsewhere. -- R.A. Heinlein, "Time Enough For Love"
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
