Thanks for the reply Hieu. The Timer returns time in seconds or milliseconds? > On Feb 20, 2016, at 1:58 AM, Hieu Hoang <[email protected]> wrote: > > it's great that you've written a new feature function but you will have to > debug it yourself. I suggest you put lots of debugging messages in your code > to find out where the problem is. > > Moses has the Timer class in /moses/Timer.h which you can use to help your > debug your problem > > Hieu Hoang > http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu> > > On 20 February 2016 at 04:20, Jasneet Sabharwal <[email protected] > <mailto:[email protected]>> wrote: > Hi Hieu, > > Just to provide more info, I had compiled moses using the following command: > "./bjam -j8 -q --with-cmph=/cs/natlang-user/jasneet/softwares/cmph-2.0/ > --with-boost=/cs/natlang-user/jasneet/softwares/boost/ --max-kenlm-order=8 -a > --with-mm --with-probing-pt”. > > Following are some more translation times from the logs using the command: > > $ grep “Translation took” mert.log > > Line 53: Translation took 9504.886 seconds total > Line 25: Translation took 16931.106 seconds total > Line 20: Translation took 17477.958 seconds total > Line 34: Translation took 18409.183 seconds total > Line 36: Translation took 20495.204 seconds total > Line 48: Translation took 16093.966 seconds total > Line 68: Translation took 4773.139 seconds total > Line 18: Translation took 22165.429 seconds total > Line 10: Translation took 23794.930 seconds total > Line 11: Translation took 26313.130 seconds total > Line 74: Translation took 6238.326 seconds total > Line 66: Translation took 14968.715 seconds total > Line 3: Translation took 28973.902 seconds total > Line 45: Translation took 27619.088 seconds total > Line 81: Translation took 4666.394 seconds total > Line 37: Translation took 36502.892 seconds total > Line 83: Translation took 3143.882 seconds total > Line 70: Translation took 20143.743 seconds total > Line 1: Translation took 38498.391 seconds total > Line 19: Translation took 39683.472 seconds total > Line 15: Translation took 39903.566 seconds total > Line 33: Translation took 40047.447 seconds total > > The times are extremely high and I’m not really sure why it is taking so much > time. > > Regards, > Jasneet >> On Feb 18, 2016, at 11:04 AM, Jasneet Sabharwal <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, >> >> I was able to solve the segmentation fault issue. It was because of OOVs. >> I’m currently trying to tune the parameters using mert, but it is running >> extremely slow. For example, from the logs: >> >> Translating: 美国 之 音 记者 伏 来 库 斯 从 布宜诺斯艾利斯 发 来 的 另 一 篇 报导 说 , 几 名 美国 国会 议员 星期二 >> 把 这 一 争论 带 到 了 布宜诺斯艾利斯 的 会议 大厅 。 >> Line 43: Initialize search took 0.007 seconds total >> Line 43: Collecting options took 0.191 seconds at moses/Manager.cpp:117 >> Line 38: Search took 1092.075 seconds >> Line 38: Decision rule took 0.000 seconds total >> Line 38: Additional reporting took 0.041 seconds total >> Line 38: Translation took 1092.132 seconds total >> >> I tried to time the functions in my feature function >> <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp> >> using clock_t but all of them show up as 0.000. I’m not sure why tuning is >> taking too much time. My moses.ini is attached in this email. >> >> Any suggestions would be helpful. >> >> Regards, >> Jasneet >> >> <moses.ini> >>> On Feb 12, 2016, at 3:58 PM, Hieu Hoang <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> I think it's >>> FeatureFunction::GetScoreProducerDescription() >>> >>> On 12/02/16 23:56, Jasneet Sabharwal wrote: >>>> Thanks, will give that a try. >>>> >>>> Also, is it possible to get the value of feature name inside the feature >>>> function. I’m specifically talking about “name” parameter in moses.ini. >>>> I’m running multiple copies of my feature function with different >>>> parameter as follows: >>>> CoarseBiLM name=CoarseBiLM tgtWordId... >>>> CoarseBiLM name=CoarseLM100 tgtWordId… >>>> CoarseBiLM name=CoarseLM1600 tgtWordId... >>>> CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId… >>>> >>>> Thanks, >>>> Jasneet >>>>> On Feb 12, 2016, at 3:39 PM, Hieu Hoang < >>>>> <mailto:[email protected]>[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> you can run the decoder >>>>> ./moses -v 3 >>>>> however, you should put debugging messages in your feature functions to >>>>> find out where the problem is. It looks like its in the Load() method so >>>>> add lots of debugging message in there and all functions it calls >>>>> >>>>> On 12/02/16 23:34, Jasneet Sabharwal wrote: >>>>>> Thanks Hieu for your reply. >>>>>> >>>>>> Is it possible to do a verbose output of what’s happening, so that I can >>>>>> identify when it’s going out of memory? I’m only running it for 1928 >>>>>> sentences. I have almost 170gb of free memory and additional 400gb >>>>>> memory in buffer. >>>>>> >>>>>> Thanks, >>>>>> Jasneet >>>>>> >>>>>>> On Feb 12, 2016, at 2:36 PM, Hieu Hoang <[email protected] >>>>>>> <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> looks like it's run out of memory. >>>>>>> >>>>>>> On 11/02/16 23:23, Jasneet Sabharwal wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I was adding a new feature function in Moses ( >>>>>>>> <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp>https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp >>>>>>>> >>>>>>>> <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp>). >>>>>>>> It works fine when I test it for 1-2 sentences, but when I’m trying >>>>>>>> to tune my parameters, I’m getting segmentation faults or sometimes it >>>>>>>> is bad_alloc. Following was one of the commands that was executed >>>>>>>> during the tuning process which caused the Segmentation Fault or >>>>>>>> bad_alloc: >>>>>>>> >>>>>>>> moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite >>>>>>>> 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758 >>>>>>>> WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered= >>>>>>>> 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303 >>>>>>>> Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= >>>>>>>> 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list >>>>>>>> run1.best100.out 100 distinct -input-file tune.word.lc.cn >>>>>>>> <http://tune.word.lc.cn/> >>>>>>>> >>>>>>>> The log is enclosed in this email. >>>>>>>> >>>>>>>> Any pointers would be very useful. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jasneet >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Moses-support mailing list >>>>>>>> [email protected] <mailto:[email protected]> >>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support> >>>>>>> >>>>>>> -- >>>>>>> Hieu Hoang >>>>>>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu> >>>>> >>>>> -- >>>>> Hieu Hoang >>>>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu> >>> >>> -- >>> Hieu Hoang >>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu> > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
