Hi Arefeh
Are your features tuneable?
I think if you run
moses --show-weights -f path-to-ini
it should tell you
cheers - Barry
On 20/08/16 15:52, arefeh kazemi wrote:
Hi Barry
Thanks so much,
moses.ini seems ok, but my feature isn't in *.best files.
These are the lines in my moses.ini:
[feature]
Aref trainFilePath=/files/entr/entr-dd-surf-tune.csv
decodeFilePath=/files/entr/entr_Tune_Decoding_WithIndex.txt
UnknownWordPenalty
....
# dense weights for feature functions
[weight-file]
/old-scratch/akazemi/experiments/entr-mymodel/dd-sur/sparse/weights.txt
This is the command that I use for tuning:
nohup nice $SCRIPTS_ROOTDIR/training/mert-moses.pl
<http://mert-moses.pl> entr-tune.en entr-tune.tr <http://entr-tune.tr> \
$MOSESCHART $WorkingDir/moses.ini --mertdir $MERTDIR \
--rootdir $SCRIPTS_ROOTDIR \
--batch-mira --return-best-dev --decoder-flags '-threads all -v 2' >
mert.out
This is a line from debug output, red ones are my features:
BEST TRANSLATION: 639354 S -> S </s> :0-0 : c=-0.024
core=(0.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0.000)
sourcePhrase=X </s> [0..28] 638176 [total=-5.857]
core=(0.000,-33.000,41.000,-34.703,-48.214,-30.033,-39.420,16.998,-96.448)
nnnnm=-3.414 prepprepm=-9.457 unalignedPenalty=79.000
cohesionPenalty=61.000 amodprepm=-10.328 detpreps=-3.297
amodamodm=-7.082 detnnm=-7.316
Best Hypothesis Generation Time: : [401.400] seconds
WRITING 100 TRANSLATION ALTERNATIVES TO run1.best100.out
N-Best Hypotheses Generation Time: : [401.645] seconds
Sentence Decoding Time: : [401.650] seconds
Translation took 1907.342 seconds
Should I check anything else?
Regards
Arefeh
On Sat, Aug 20, 2016 at 4:53 PM, Barry Haddow
<[email protected] <mailto:[email protected]>> wrote:
Hi Arefeh
Attached.
If you look at the files produced in your tuning run, you should
see the following produced for each iteration:
run2.best100.out.gz
run2.dense
run2.extract.err
run2.extract.out
run2.features.dat
run2.init.opt
run2.mert.log
run2.mert.out
run2.mira.out
run2.moses.ini
run2.out
run2.scores.dat
run2.sparse-weights
run2.weights.txt
(run1 may be different)
In run2.best100.out.gz you should see the sparse feature values.
In run2.mert.out you should see the sparse feature weights.
cheers - Barry
On 20/08/16 12:47, arefeh kazemi wrote:
Hi Barry
Thanks,
I tried verbose and I can see my sparse feature and it's scores
for each sentence, but the weights.txt file is still empty.
(I use kbmira for tuning).
I think there is a problem with moses.ini file.
Could you please send an example .ini file with a sparse feature?
Thanks
Arefeh
On Thu, Aug 18, 2016 at 6:13 PM, Barry Haddow
<[email protected] <mailto:[email protected]>>
wrote:
Hi Arefeh
The quickest way to see if Moses is using your feature is to
put a debug message in it to see if it gets called. You can
also increase the debug of Moses (try -v 2) to see if your
feature's scores appear in the breakdown.
To populate the weights file, you will need to run tuning
(kbmira or pro). If you just decode with an empty weights
file, all the weights will be set to 0 and Moses will not
update the file,
cheers - Barry
On 18/08/16 14:15, arefeh kazemi wrote:
Hi Barry
Thanks.
I create an empty weights.txt file and write it's address in
moses.ini. Moses runs normally but weights file remains
empty. It seems moses doesn't use my feature.
Regards
Arefeh
On Wed, Aug 17, 2016 at 12:57 PM, Barry Haddow
<[email protected]
<mailto:[email protected]>> wrote:
Hi Arefeh
That seems OK. Tuning (with kbmira or pro) will create a
weights file for the sparse features, which you can add
with:
[weight-file]
/path/to/sparse/weights
What goes wrong when you run moses?
cheers - Barry
On 17/08/16 07:50, arefeh kazemi wrote:
Hi
This is just a kindly reminder that I'm waiting for
response.
Thanks
Arefe
---------- Forwarded message ----------
From: *arefeh kazemi* <[email protected]
<mailto:[email protected]>>
Date: Tue, Aug 9, 2016 at 7:28 PM
Subject: moses.ini file for sparse features
To: Moses-support <[email protected]
<mailto:[email protected]>>
Hi
I've implemented a sparse feature function in Moses
Hiero system but I don't know what are the parameters
in moses.ini file for a sparse feature.
for the dense version of my feature, I had these lines
in my ini file:
[features]
Aref num-features=4 ...
[weight]
Aref= 0.2 0.2 0.2 0.2
Now, What should I write for my sparse feature in .ini
file? I have removed the weights for Aref and also
"num-features" from ini file, but it doesn't work.
Regards
Arefeh Kazemi
--
Arefeh Kazemi
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
The University of Edinburgh is a charitable body,
registered in Scotland, with registration number SC005336.
--
Arefeh Kazemi
The University of Edinburgh is a charitable body, registered
in Scotland, with registration number SC005336.
--
Arefeh Kazemi
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
Arefeh Kazemi
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
