Hi, Alright, my phrase table is just 100MB compressed. And after taking quite long to read the phrase table, i got an exception bad_alloc. I have quite a bit of RAM, so, i am not sure what I am doing wrong .
Start loading text SCFG phrase table. Moses format : [51.000] seconds Reading /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 **************************************************************************************************** Exception: std::bad_alloc Are the changes in the master branch mostly related to an updated moses file format ? Are there any changes in the algorithms(mert/ibm models) that warrant a re-run of tests done using the previous version ? On Mon, Sep 9, 2013 at 1:24 PM, rohit dholakia <[email protected]> wrote: > Hi > > I have recently managed to install moses from the master branch on > github. The file format has changed. The training and mert has gone > without incident. But, i have observed that reading the phrase table is > taking very very long compared to the previous moses. From mert.out and > from translating test data log, > > *Loading table into memory...done.* > *Start loading text SCFG phrase table. Moses format : [51.000] seconds* > *Reading > /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz* > * > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > * > > > Now, this step, this step is taking really long. Am I the only one who is > observing this ? My phrase table is trained on only 110K sentences, quite > noisy ones at that. Has only 4.8 million rules. > > *[rd@seawolf1 TestingMoses]$ zcat train/model/phrase-table.gz | wc -l* > *4838810* > > To explain the time taken, its been more than 5 minutes and not even a > single star has come up below that last line ( ---- 5 --- 10 ) . > > Thanks. > > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
