Re: [Moses-support] RNNLM Integration?

2016-03-07 Thread Marcin Junczys-Dowmunt
Something like this maybe? http://www.statmt.org/wmt15/pdf/WMT34.pdf W dniu 07.03.2016 o 21:41, Lane Schwartz pisze: Philipp, Are you aware of any published work examining the importance of hypothesis recombination in terms of time/space/quality tradeoffs? Lane On Mon, Mar 7, 2016 at 3:19

Re: [Moses-support] What is sentence alignment score in training step 2 RUN GIZA++

2016-03-07 Thread Philipp Koehn
Hi, this ought to be the score for p(a|e,f) (or p(e,a|f)?) according the the last IBM Model that was run - typically IBM Model 4. The IBM Model 4 formula is quite complex. It considers word alignment, fertility, reordering, etc. You can find it here: Peter F. Brown and Stephen A. Della-Pietra and

Re: [Moses-support] Error in Running the Language Model Training of the KenLM

2016-03-07 Thread Marcin Junczys-Dowmunt
Try -S 50% or even smaller. But 4GB is awfully small. W dniu 07.03.2016 o 21:54, BIRENDRA CHAUHAN SINGH pisze: on running this: mkdir ~/lm cd ~/lm ~/mosesdecoder/bin/lmplz -o 3 <~/corpus/news-commentary-v8.fr-en.true.en > news-commentary-v8.fr-en.arpa.en Error: bhupendra@berry:~/lm$ ~/moses

[Moses-support] Error in Running the Language Model Training of the KenLM

2016-03-07 Thread BIRENDRA CHAUHAN SINGH
on running this: mkdir ~/lm cd ~/lm ~/mosesdecoder/bin/lmplz -o 3 <~/corpus/news-commentary-v8.fr-en.true.en > news-commentary-v8.fr-en.arpa.en Error: bhupendra@berry:~/lm$ ~/mosesdecoder/bin/lmplz -o 3 <~/corpus/news-commentary-v8.fr-en.true.en > news-commentary-v8.fr-en.arpa.en === 1/5 Coun

Re: [Moses-support] RNNLM Integration?

2016-03-07 Thread Lane Schwartz
Philipp, Are you aware of any published work examining the importance of hypothesis recombination in terms of time/space/quality tradeoffs? Lane On Mon, Mar 7, 2016 at 3:19 PM, Philipp Koehn wrote: > Hi, > > integrating this into the decoder will break all hypothesis recombination, > so it ma

Re: [Moses-support] RNNLM Integration?

2016-03-07 Thread Philipp Koehn
Hi, integrating this into the decoder will break all hypothesis recombination, so it may be better (and definitely easier) to use the RNNLM to rerank n-best lists. -phi On Mon, Mar 7, 2016 at 3:46 PM, Jake Ballinger wrote: > Hello everyone, > > Has anyone used an RNNLM language model instead o

Re: [Moses-support] Training: ferility limit

2016-03-07 Thread Philipp Koehn
Hi, it generally makes little sense to align a sentence with 1 word to a sentence with 28 words. It may be possible to change the hard-coded limit to 9 words, but for all practical purposes it is better to filter out sentence pairs that violate exceed the default ratio. -phi On Mon, Mar 7, 2016

[Moses-support] RNNLM Integration?

2016-03-07 Thread Jake Ballinger
Hello everyone, Has anyone used an RNNLM language model instead of one of the recommended language models? I was specifically looking at the RNNLM toolkit provided by Tomas Mikolov at http://rnnlm.org/. Thank you! -- Jake Ballinger Major: Computer Science Minors: Chinese, French, Spanish, & Mat

Re: [Moses-support] In-domain dictionary - where to include it

2016-03-07 Thread Philipp Koehn
Hi, it is likely that it did not run the decoder et al. Check the STDERR or STDOUT for the command in which the moses decoder was invoked. Run just that command manually and check for error messages. -phi On Sun, Mar 6, 2016 at 1:18 AM, Joze Kadivec wrote: > Hello, > > > > Can you please help

[Moses-support] Reordering Constraints with "zone" and "wall" markup

2016-03-07 Thread Patrik Lambert
Hi, I am trying to specify reordering constraints with the tag and it does not work as I would expect. The output corresponding to an input sentence I translated from English to Spanish is as follows: input: My favourite hotel in Cologne output: mi |0-0| hotel favorito |1-2| en Colonia |3-4|

[Moses-support] Training: ferility limit

2016-03-07 Thread Masa Taka
> the maximum allowed limit for a source word fertility > source length = 1 target length = 28 ratio 28 ferility limit : 9 With GIZA++ training, we have found some warning as above (in the attached .txt file.) This mailing list archive says the following and I know what and why. https://www.mail-

Re: [Moses-support] Missing /usr/lib64/lib64/libboost_thread-mt.so.5 for multithreaded GIZA

2016-03-07 Thread Hieu Hoang
where did you get the code for mgiza from and what was the exact commands you used to compile? I just tested mgiza compilation with the code from https://github.com/moses-smt/mgiza using the commands cmake . cmake . make -j4 It seems to compile ok On 06/03/2016 20:02, Sergey A. wrot