| Hi,
I was able to solve the segmentation fault issue. It was because of OOVs. I’m currently trying to tune the parameters using mert, but it is running extremely slow. For example, from the logs:
Translating: 美国 之 音 记者 伏 来 库 斯 从 布宜诺斯艾利斯 发 来 的 另 一 篇 报导 说 , 几 名 美国 国会 议员 星期二 把 这 一 争论 带 到 了 布宜诺斯艾利斯 的 会议 大厅 。 Line 43: Initialize search took 0.007 seconds total Line 43: Collecting options took 0.191 seconds at moses/Manager.cpp:117 Line 38: Search took 1092.075 seconds Line 38: Decision rule took 0.000 seconds total Line 38: Additional reporting took 0.041 seconds total Line 38: Translation took 1092.132 seconds total
I tried to time the functions in my feature function using clock_t but all of them show up as 0.000. I’m not sure why tuning is taking too much time. My moses.ini is attached in this email.
Any suggestions would be helpful.
Regards, Jasneet
|
moses.ini
Description: Binary data
I think it's
FeatureFunction::GetScoreProducerDescription()
On 12/02/16 23:56, Jasneet Sabharwal
wrote:
Thanks, will give that a try.
Also, is it possible to get the value of feature
name inside the feature function. I’m specifically talking about
“name” parameter in moses.ini. I’m running multiple copies of my
feature function with different parameter as follows:
CoarseBiLM name=CoarseBiLM tgtWordId...
CoarseBiLM name=CoarseLM100 tgtWordId…
CoarseBiLM name=CoarseLM1600 tgtWordId...
CoarseBiLM
name=CoarseBiLMWithoutClustering tgtWordId…
Thanks, Jasneet
you can
run the decoder
./moses -v 3
however, you should put debugging messages in your
feature functions to find out where the problem is. It
looks like its in the Load() method so add lots of
debugging message in there and all functions it calls
On 12/02/16 23:34,
Jasneet Sabharwal wrote:
Thanks Hieu for your reply.
Is it possible to do a verbose output
of what’s happening, so that I can identify when
it’s going out of memory? I’m only running it for
1928 sentences. I have almost 170gb of free memory
and additional 400gb memory in buffer.
Thanks,
Jasneet
looks like it's run out of
memory.
On 11/02/16
23:23, Jasneet Sabharwal wrote:
Hi,
I was adding a new
feature function in Moses ( https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp).
It works fine when I test it for 1-2
sentences, but when I’m trying to
tune my parameters, I’m getting
segmentation faults or sometimes it
is bad_alloc. Following was one of
the commands that was executed
during the tuning process which
caused the Segmentation Fault or
bad_alloc:
moses -threads 40 -v 0
-config filtered/moses.ini
-weight-overwrite 'CoarseLM100=
0.075758 LM0= 0.075758
CoarseBiLMNotClustered= 0.075758
WordPenalty0= -0.151515
PhrasePenalty0= 0.030303
CoarseBiLMClustered= 0.075758
TranslationModel0= 0.030303 0.030303
0.030303 0.030303 Distortion0=
0.045455 CoarseLM1600= 0.075758
LexicalReordering0= 0.045455
0.045455 0.045455 0.045455 0.045455
0.045455' -n-best-list
run1.best100.out 100 distinct
-input-file tune.word.lc.cn
The log is enclosed in
this email.
Any pointers would be
very useful.
Thanks,
Jasneet
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
--
Hieu Hoang
http://www.hoang.co.uk/hieu
--
Hieu Hoang
http://www.hoang.co.uk/hieu
|
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
