Thanks Mathias, I'll try these out. ----- Original Message ----- From: "Matthias Huck" <[email protected]> To: "Hieu Hoang" <[email protected]>, "Jasneet Sabharwal" <[email protected]> Cc: "moses-support" <[email protected]> Sent: Saturday, February 20, 2016 6:08:52 PM Subject: Re: [Moses-support] Segmentation Fault
Hi Jasneet, Why don't you use a proper profiling tool, e.g. the one in valgrind [1]? If you visualize its output [2], you'll see quickly where the program spends all the computing time. Cheers, Matthias [1] http://valgrind.org/docs/manual/cl-manual.html [2] https://github.com/jrfonseca/gprof2dot On Sat, 2016-02-20 at 09:58 +0000, Hieu Hoang wrote: > it's great that you've written a new feature function but you will > have to debug it yourself. I suggest you put lots of debugging > messages in your code to find out where the problem is. > > Moses has the Timer class in /moses/Timer.h which you can use to help > your debug your problem > > Hieu Hoang > http://www.hoang.co.uk/hieu > > On 20 February 2016 at 04:20, Jasneet Sabharwal < > [email protected]> wrote: > > Hi Hieu, > > > > Just to provide more info, I had compiled moses using the following > > command: "./bjam -j8 -q --with-cmph=/cs/natlang > > -user/jasneet/softwares/cmph-2.0/ --with-boost=/cs/natlang > > -user/jasneet/softwares/boost/ --max-kenlm-order=8 -a --with-mm - > > -with-probing-pt”. > > > > Following are some more translation times from the logs using the > > command: > > > > $ grep “Translation took” mert.log > > > > Line 53: Translation took 9504.886 seconds total > > Line 25: Translation took 16931.106 seconds total > > Line 20: Translation took 17477.958 seconds total > > Line 34: Translation took 18409.183 seconds total > > Line 36: Translation took 20495.204 seconds total > > Line 48: Translation took 16093.966 seconds total > > Line 68: Translation took 4773.139 seconds total > > Line 18: Translation took 22165.429 seconds total > > Line 10: Translation took 23794.930 seconds total > > Line 11: Translation took 26313.130 seconds total > > Line 74: Translation took 6238.326 seconds total > > Line 66: Translation took 14968.715 seconds total > > Line 3: Translation took 28973.902 seconds total > > Line 45: Translation took 27619.088 seconds total > > Line 81: Translation took 4666.394 seconds total > > Line 37: Translation took 36502.892 seconds total > > Line 83: Translation took 3143.882 seconds total > > Line 70: Translation took 20143.743 seconds total > > Line 1: Translation took 38498.391 seconds total > > Line 19: Translation took 39683.472 seconds total > > Line 15: Translation took 39903.566 seconds total > > Line 33: Translation took 40047.447 seconds total > > > > The times are extremely high and I’m not really sure why it is > > taking so much time. > > > > Regards, > > Jasneet > > > On Feb 18, 2016, at 11:04 AM, Jasneet Sabharwal < > > > [email protected]> wrote: > > > > > > Hi, > > > > > > I was able to solve the segmentation fault issue. It was because > > > of OOVs. I’m currently trying to tune the parameters using mert, > > > but it is running extremely slow. For example, from the logs: > > > > > > Translating: 美国 之 音 记者 伏 来 库 斯 从 布宜诺斯艾利斯 发 来 的 另 一 篇 报导 说 , 几 名 > > > 美国 国会 议员 星期二 把 这 一 争论 带 到 了 布宜诺斯艾利斯 的 会议 大厅 。 > > > Line 43: Initialize search took 0.007 seconds total > > > Line 43: Collecting options took 0.191 seconds at > > > moses/Manager.cpp:117 > > > Line 38: Search took 1092.075 seconds > > > Line 38: Decision rule took 0.000 seconds total > > > Line 38: Additional reporting took 0.041 seconds total > > > Line 38: Translation took 1092.132 seconds total > > > > > > I tried to time the functions in my feature function using > > > clock_t but all of them show up as 0.000. I’m not sure why tuning > > > is taking too much time. My moses.ini is attached in this email. > > > > > > Any suggestions would be helpful. > > > > > > Regards, > > > Jasneet > > > > > > <moses.ini> > > > > On Feb 12, 2016, at 3:58 PM, Hieu Hoang <[email protected]> > > > > wrote: > > > > > > > > I think it's > > > > FeatureFunction::GetScoreProducerDescription() > > > > > > > > On 12/02/16 23:56, Jasneet Sabharwal wrote: > > > > > Thanks, will give that a try. > > > > > > > > > > Also, is it possible to get the value of feature name inside > > > > > the feature function. I’m specifically talking about “name” > > > > > parameter in moses.ini. I’m running multiple copies of my > > > > > feature function with different parameter as follows: > > > > > CoarseBiLM name=CoarseBiLM tgtWordId... > > > > > CoarseBiLM name=CoarseLM100 tgtWordId… > > > > > CoarseBiLM name=CoarseLM1600 tgtWordId... > > > > > CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId… > > > > > > > > > > Thanks, > > > > > Jasneet > > > > > > On Feb 12, 2016, at 3:39 PM, Hieu Hoang < > > > > > > [email protected]> wrote: > > > > > > > > > > > > you can run the decoder > > > > > > ./moses -v 3 > > > > > > however, you should put debugging messages in your feature > > > > > > functions to find out where the problem is. It looks like > > > > > > its in the Load() method so add lots of debugging message > > > > > > in there and all functions it calls > > > > > > > > > > > > On 12/02/16 23:34, Jasneet Sabharwal wrote: > > > > > > > Thanks Hieu for your reply. > > > > > > > > > > > > > > Is it possible to do a verbose output of what’s > > > > > > > happening, so that I can identify when it’s going out of > > > > > > > memory? I’m only running it for 1928 sentences. I have > > > > > > > almost 170gb of free memory and additional 400gb memory > > > > > > > in buffer. > > > > > > > > > > > > > > Thanks, > > > > > > > Jasneet > > > > > > > > > > > > > > > On Feb 12, 2016, at 2:36 PM, Hieu Hoang < > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > looks like it's run out of memory. > > > > > > > > > > > > > > > > On 11/02/16 23:23, Jasneet Sabharwal wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I was adding a new feature function in Moses (https:/ > > > > > > > > > /github.com/KonceptGeek/mosesdecoder/blob/master/mose > > > > > > > > > s/FF/CoarseBiLM.cpp). It works fine when I test it > > > > > > > > > for 1-2 sentences, but when I’m trying to tune my > > > > > > > > > parameters, I’m getting segmentation faults or > > > > > > > > > sometimes it is bad_alloc. Following was one of the > > > > > > > > > commands that was executed during the tuning process > > > > > > > > > which caused the Segmentation Fault or bad_alloc: > > > > > > > > > > > > > > > > > > moses -threads 40 -v 0 -config filtered/moses.ini > > > > > > > > > -weight-overwrite 'CoarseLM100= 0.075758 LM0= > > > > > > > > > 0.075758 CoarseBiLMNotClustered= 0.075758 > > > > > > > > > WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 > > > > > > > > > CoarseBiLMClustered= 0.075758 TranslationModel0= > > > > > > > > > 0.030303 0.030303 0.030303 0.030303 Distortion0= > > > > > > > > > 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= > > > > > > > > > 0.045455 0.045455 0.045455 0.045455 0.045455 > > > > > > > > > 0.045455' -n-best-list run1.best100.out 100 distinct > > > > > > > > > -input-file tune.word.lc.cn > > > > > > > > > > > > > > > > > > The log is enclosed in this email. > > > > > > > > > > > > > > > > > > Any pointers would be very useful. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Jasneet > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > Moses-support mailing list > > > > > > > > > [email protected] > > > > > > > > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > -- > > > > > > > > Hieu Hoang > > > > > > > > http://www.hoang.co.uk/hieu > > > > > > -- > > > > > > Hieu Hoang > > > > > > http://www.hoang.co.uk/hieu > > > > -- > > > > Hieu Hoang > > > > http://www.hoang.co.uk/hieu > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
