My bad, figured the problem. I had added the *sa_suffix file extension in my filtering command. It works fine.
On Wed, Oct 19, 2016 at 10:42 AM, Sameer Bhadouria < [email protected]> wrote: > > Hello, > > I am trying to prune my phrase table using the SALM generated suffix array > indices but I don't see anything pruned. I am using the > news-commentary-v10 data for english to french translation. > > I followed the instructions on this wiki: > http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc5 > > Here is the command I used to run this: > > I generated the suffix arrays using: > > SALM/Bin/Linux/Index/IndexSA.O32 TARGET > SALM/Bin/Linux/Index/IndexSA.O32 SOURCE > > > [sbhadour@sbhadour-ld1 model]$ cat phrase-table | /home/sbhadour/work/ > experiments/mosesdecoder/contrib/sigtest-filter/filter-pt -e > /home/sbhadour/work/experiments/lang-training- > data/salm-sufix-arr-index/en_fr/news-commentary-v10.fr-en.clean.fr.sa_suffix > -f /home/sbhadour/work/experiments/lang-training- > data/salm-sufix-arr-index/en_fr/news-commentary-v10.fr-en.clean.en.sa_suffix > -l 'a+e -n 30' > phrase-table.pruned > > -l = a+e -n 30 > > Filtering using P(e|f) only. n=0 > > ..................................................[n:500000] > ..................................................[n:1000000] > ..................................................[n:1500000] > ..................................................[n:2000000] > ..................................................[n:2500000] > ..................................................[n:3000000] > ..................................................[n:3500000] > ..................................................[n:4000000] > ..................................................[n:4500000] > ..................................................[n:5000000] > ..................................................[n:5500000] > ..................................................[n:6000000] > ............ > > ------------------------------------------------------ > > unfiltered phrases pairs: 6121925 > P(f|e) filter [first]: 0 (0%) > significance filter: 0 (0%) > TOTAL FILTERED: 0 (0%) > FILTERED phrase pairs: 6121925 (100%) > > ------------------------------------------------------ > > > Am I missing anything? Help appreciated! > > -- > Regards, > Sameer Bhadouria. > -- Regards, Sameer Bhadouria.
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
