it's great that you've written a new feature function but you will have to debug it yourself. I suggest you put lots of debugging messages in your code to find out where the problem is.
Moses has the Timer class in /moses/Timer.h which you can use to help your debug your problem Hieu Hoang http://www.hoang.co.uk/hieu On 20 February 2016 at 04:20, Jasneet Sabharwal <jasneet.sabhar...@sfu.ca> wrote: > Hi Hieu, > > Just to provide more info, I had compiled moses using the following > command: "./bjam -j8 -q > --with-cmph=/cs/natlang-user/jasneet/softwares/cmph-2.0/ > --with-boost=/cs/natlang-user/jasneet/softwares/boost/ --max-kenlm-order=8 > -a --with-mm --with-probing-pt”. > > Following are some more translation times from the logs using the command: > > $ grep “Translation took” mert.log > > Line 53: Translation took 9504.886 seconds total > Line 25: Translation took 16931.106 seconds total > Line 20: Translation took 17477.958 seconds total > Line 34: Translation took 18409.183 seconds total > Line 36: Translation took 20495.204 seconds total > Line 48: Translation took 16093.966 seconds total > Line 68: Translation took 4773.139 seconds total > Line 18: Translation took 22165.429 seconds total > Line 10: Translation took 23794.930 seconds total > Line 11: Translation took 26313.130 seconds total > Line 74: Translation took 6238.326 seconds total > Line 66: Translation took 14968.715 seconds total > Line 3: Translation took 28973.902 seconds total > Line 45: Translation took 27619.088 seconds total > Line 81: Translation took 4666.394 seconds total > Line 37: Translation took 36502.892 seconds total > Line 83: Translation took 3143.882 seconds total > Line 70: Translation took 20143.743 seconds total > Line 1: Translation took 38498.391 seconds total > Line 19: Translation took 39683.472 seconds total > Line 15: Translation took 39903.566 seconds total > Line 33: Translation took 40047.447 seconds total > > The times are extremely high and I’m not really sure why it is taking so > much time. > > Regards, > Jasneet > > On Feb 18, 2016, at 11:04 AM, Jasneet Sabharwal <jasneet.sabhar...@sfu.ca> > wrote: > > Hi, > > I was able to solve the segmentation fault issue. It was because of OOVs. > I’m currently trying to tune the parameters using mert, but it is running > extremely slow. For example, from the logs: > > Translating: 美国 之 音 记者 伏 来 库 斯 从 布宜诺斯艾利斯 发 来 的 另 一 篇 报导 说 , 几 名 美国 国会 议员 > 星期二 把 这 一 争论 带 到 了 布宜诺斯艾利斯 的 会议 大厅 。 > Line 43: Initialize search took 0.007 seconds total > Line 43: Collecting options took 0.191 seconds at moses/Manager.cpp:117 > Line 38: Search took 1092.075 seconds > Line 38: Decision rule took 0.000 seconds total > Line 38: Additional reporting took 0.041 seconds total > Line 38: Translation took 1092.132 seconds total > > I tried to time the functions in my feature function > <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp> > using > clock_t but all of them show up as 0.000. I’m not sure why tuning is taking > too much time. My moses.ini is attached in this email. > > Any suggestions would be helpful. > > Regards, > Jasneet > > <moses.ini> > > On Feb 12, 2016, at 3:58 PM, Hieu Hoang <hieuho...@gmail.com> wrote: > > I think it's > FeatureFunction::GetScoreProducerDescription() > > On 12/02/16 23:56, Jasneet Sabharwal wrote: > > Thanks, will give that a try. > > Also, is it possible to get the value of feature name inside the feature > function. I’m specifically talking about “name” parameter in moses.ini. I’m > running multiple copies of my feature function with different parameter as > follows: > > CoarseBiLM name=CoarseBiLM tgtWordId... > CoarseBiLM name=CoarseLM100 tgtWordId… > CoarseBiLM name=CoarseLM1600 tgtWordId... > CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId… > > Thanks, > Jasneet > > On Feb 12, 2016, at 3:39 PM, Hieu Hoang < <hieuho...@gmail.com> > hieuho...@gmail.com> wrote: > > you can run the decoder > ./moses -v 3 > however, you should put debugging messages in your feature functions to > find out where the problem is. It looks like its in the Load() method so > add lots of debugging message in there and all functions it calls > > On 12/02/16 23:34, Jasneet Sabharwal wrote: > > Thanks Hieu for your reply. > > Is it possible to do a verbose output of what’s happening, so that I can > identify when it’s going out of memory? I’m only running it for 1928 > sentences. I have almost 170gb of free memory and additional 400gb memory > in buffer. > > Thanks, > Jasneet > > On Feb 12, 2016, at 2:36 PM, Hieu Hoang <hieuho...@gmail.com> wrote: > > looks like it's run out of memory. > > On 11/02/16 23:23, Jasneet Sabharwal wrote: > > Hi, > > I was adding a new feature function in Moses ( > <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp> > https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp). > It works fine when I test it for 1-2 sentences, but when I’m trying to tune > my parameters, I’m getting segmentation faults or sometimes it is > bad_alloc. Following was one of the commands that was executed during the > tuning process which caused the Segmentation Fault or bad_alloc: > > moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite > 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758 > WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered= > 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303 > Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= 0.045455 > 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list run1.best100.out > 100 distinct -input-file tune.word.lc.cn > > The log is enclosed in this email. > > Any pointers would be very useful. > > Thanks, > Jasneet > > > > > > _______________________________________________ > Moses-support mailing > listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support > > > -- > Hieu Hoanghttp://www.hoang.co.uk/hieu > > > > -- > Hieu Hoanghttp://www.hoang.co.uk/hieu > > > > -- > Hieu Hoanghttp://www.hoang.co.uk/hieu > > > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
