Hui, the wrapper script really just exists, because SRILM (and the wrapper) sets the name of the produced LM file with "-lm" and lmplz sets it with "-arpa". If you allow as an alternative name for the switch "-lm", I'll remove it.
-phi On Tue, Feb 24, 2015 at 8:39 PM, Kenneth Heafield <[email protected]> wrote: > Try removing this bit of text and just calling the lmplz binary > directly. It's not clear to me why that wrapper script still exists. > > $moses-script-dir/ems/support/lmplz-wrapper.perl -bin > > > -------- Forwarded Message -------- > Subject: Re: [Moses-support] BadDiscountException > Date: Tue, 24 Feb 2015 06:16:43 -0800 > From: fatma elzahraa Eltaher <[email protected]> > To: Kenneth Heafield <[email protected]> > > > > I use kenlm model and when try to add --discount_fallback=1 for setting > I get this error Unknown option: discount_fallback. > I attached config.toy where must I change to solve this problem ? > > > thank you, > > > > Fatma El-Zahraa El -Taher > > Teaching Assistant at Computer & System department > > Faculty of Engineering, Azhar University > > Email : [email protected] <mailto:[email protected]> > mobile: +201141600434 > > > On Tue, Feb 24, 2015 at 5:22 AM, Kenneth Heafield <[email protected] > <mailto:[email protected]>> wrote: > > The closed-form estimates for Kneser-Ney are not well-defined on toy or > class-based data. I recommend using more training data. If this is a > class-based model, pass --discount_fallback. > > Kenneth > > On 02/24/2015 08:04 AM, fatma elzahraa Eltaher wrote: > > Dears, > > I get the following error in LM_toy_train.65.STDERR: > > Unigram tokens 25188 types 39 > > === 2/5 Calculating and sorting adjusted counts === > > Chain sizes: 1:468 2:322921696 3:605478272 4:968765120 5:1412782592 > > /home/fatma/Desktop/Folder/mosesdecoder/lm/builder/adjust_counts.cc:50 > > in void > > lm::builder::{anonymous}::StatCollector::CalculateDiscounts(const > > lm::builder::DiscountConfig&) threw BadDiscountException because > `s.n[j] > > == 0'. > > Could not calculate Kneser-Ney discounts for 1-grams with adjusted > count > > 4 because we didn't observe any 1-grams with adjusted count 3; Is this > > small or artificial data? > > How do I fix it? > > > > > > thank you, > > > > > > > > Fatma El-Zahraa El -Taher > > > > Teaching Assistant at Computer & System department > > > > Faculty of Engineering, Azhar University > > > > Email : [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > > mobile: +201141600434 > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
