Hi Prashant
PRO also assumes that the model score depends linearly on the weights,
as do the mira variants. There have been some attempts at "tuning" of
hyperparameters, but it's a tricky problem. For example, see:
http://books.nips.cc/papers/files/nips25/NIPS2012_0650.pdf (for parsing)
http://www.aclweb.org/anthology/W13-2229.pdf
cheers - Barry
On 17/01/14 18:08, Hieu Hoang wrote:
interesting. I don't think you can use MERT because it assumes that
there is a linear relationship between weights and model scores.
However, I think you can use PRO.
I added a feature function
HyperParameterAsWeight
that uses it's weights to change some hyper-parameters such as stack
size and beam width.
It's a hack, but I hope it's a start of what you want to do. Change it
however you want. I haven't tested it so it's up to you to test it
Fyi, the commit is here
https://github.com/moses-smt/mosesdecoder/commit/20e7d078a6ed23e6a293801955ba09e622f993df
On 17 January 2014 16:10, Prashant Mathur <[email protected]
<mailto:[email protected]>> wrote:
Hi All,
I am tuning the standard models with an extended feature.
As far as I understood, we can tune only the feature weights in mert.
In my case I have some more hyper parameters that I want to tune
together with the feature weights .. for example. feature learning
rate.
Once the nbest-list is created these parameters cannot have any effect
on the list, but they are useful for generating a better nbest-list.
Is there a way to tune all the parameters and hyper parameters
together?
Thanks,
Prashant
PS: Till now I was tuning these hyper parameters using Simplex.
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
