Hi all, I am running experiments on Bilingual Neural LM.
For extract_training.py, I set --prune-target-vocab 10000 --prune-source-vocab 10000 --target-context 5 --source-context 4 For train_nplm.py, I set --ngram-size 14 --output-embedding 512 --input-embedding 192 --hidden 512 --e 5 I use 2 million parallel sentence pairs for training. The implementation is from https://github.com/rsennrich/nplm >From the training log generated by train_nplm.py at the first 2 iterations, I have Number of training minibatches: 52853 Epoch 1 Current learning rate: 1 Training minibatches: 10000...20000...30000...40000...50000...done. Training NCE log-likelihood: -1.64277e+08 Writing model Epoch 2 Current learning rate: 1 Training minibatches: 10000...20000...30000...40000...50000...done. Training NCE log-likelihood: -1.38122e+08 Writing model The NCE log-likelihood number is suspicious. It is very low. Did I set any parameters wrong? Regards, Jian -- Jian Zhang Centre for Next Generation Localisation (CNGL) <http://www.cngl.ie/index.html> Dublin City University <http://www.dcu.ie/>
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
