Hi, Thanks Hieu for your reply. I have used *CreateOnDiskPt* to binarize the model and stored in *phrase-table.3.folder *using following command:
*~/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2 phrase-table.3.gz phrase-table.3.folder* Also I have made changes in *moses.ini.3 (*i.e. I have converted *PhraseDictionaryMemory* to *PhraseDictionaryCompact* as follows*) * PhraseDictionaryOnDisk name=TranslationModel0 num-features=4 path=/home/shubham/models/fr-en/phrase-table.3.folder input-factor=0 output-factor=0 Now, when I run it using :* ~/mosesdecoder/bin/moses -f moses.ini.3 * , it gave following error after *Created input-output object*: *terminate called after throwing an instance of 'util::Exception'* * what(): OnDiskPt/OnDiskWrapper.cpp:217 in uint64_t OnDiskPt::OnDiskWrapper::GetMisc(const string&) const threw util::Exception because `iter == m_miscInfo.end()'.* *Couldn't find value for key NumSourceFactors* *Aborted (core dumped)* Here, I do not know that what key value should I pass and how ? Can you please help me in this regard. Thank you so much for your help. Regards, Shubham On Fri, Dec 9, 2016 at 4:27 PM, Hieu Hoang <[email protected]> wrote: > This is a hierarchical model. You must binarize with CreateOnDiskPt for > this model > > Hieu Hoang > http://www.hoang.co.uk/hieu > > On 9 December 2016 at 08:18, Shubham Khandelwal <[email protected]> > wrote: > >> Hello, >> >> Thanks. It worked now. I have created compact phrase table. >> Now, when I am running it using following command: >> >> *~/mosesdecoder/bin/moses >> -f ~/Translate/models/de-en/model/moses.ini.2 -threads all* >> >> Then, after creating input-output object, it gave following segmentation >> fault error: >> >> Created input-output object : [14.796] seconds >> Ich bin ein Student >> Line 0: Initialize search took 0.000 seconds total >> Translating: <s> Ich bin ein Student </s> ||| [0,0]=X (1) [0,1]=X (1) >> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1) [1,2]=X (1) >> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1) [2,4]=X (1) >> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1) [4,5]=X (1) >> [5,5]=X (1) >> >> Segmentation fault (core dumped) >> >> In my machine, I have 40GB RAM but still I am confused why it gave this >> error. >> Can you please help me in this regard. I have attached moses.ini.2 for >> your reference. >> >> Thanks. >> >> Regards, >> Shubham >> >> On Fri, Dec 9, 2016 at 2:02 AM, Hieu Hoang <[email protected]> wrote: >> >>> maybe try >>> >>> -encoding None >>> >>> On 08/12/2016 19:44, Shubham Khandelwal wrote: >>> >>> Hi Hieu, >>> >>> Thanks for your reply. >>> Yes, I have used the absolute path and also I tried with -T but it did >>> not work. >>> Is there any other solution to this problem. >>> >>> Btw, Can anybody please upload the compact model of all pre-made models >>> as this will take less space and also it will be very fast during decoding. >>> >>> Thanks. >>> >>> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <[email protected]> wrote: >>> >>>> the previous email you referred to says that the directory >>>> >>>> * binarised-model/ * >>>> >>>> *must exist before you run it, otherwise it will segfault. I would also >>>> use absolute path to make sure, ie. not * >>>> *binarised-model/phrase-table * >>>> >>>> *but * >>>> >>>> >>>> * /home/shubham/moses/binarised-model/phrase-table * >>>> >>>> *The previous email exchange also says you should try to add the >>>> argument * >>>> >>>> >>>> * -T . * >>>> >>>> Hieu Hoang >>>> http://www.hoang.co.uk/hieu >>>> >>>> On 8 December 2016 at 15:52, Shubham Khandelwal <[email protected]> >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> This is just the reminder of my previous email. >>>>> >>>>> Thanking You. >>>>> >>>>> Regards, >>>>> Shubham >>>>> >>>>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I have just downloaded phrase-table.2.gz (18GB) de-en model >>>>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made >>>>>> models. >>>>>> Now, I am converting them to PhraseDictionaryCompact using following >>>>>> command (for exmaple): >>>>>> >>>>>> >>>>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in >>>>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table >>>>>> * >>>>>> >>>>>> But after passing 1/3, it gave following segementation fault error: >>>>>> >>>>>> *Pass 1/3: Creating hash function for rank assignment* >>>>>> *Segmentation fault (core dumped)* >>>>>> >>>>>> I have found almost same issue on this thread: >>>>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033 >>>>>> However, I have provided the existing *binarised-model *folder in >>>>>> the command. Also, I have the write-access in /tmp but still it gave >>>>>> sementation fault. >>>>>> >>>>>> Can you please tell me what could be wrong here ? >>>>>> >>>>>> Thanking You. >>>>>> >>>>>> Regards, >>>>>> Shubham >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>>>> >>>> >>> >>> >>> >>> >> >> >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
