strange, did the CreateOnDiskPt command execute ok, ie. with no error? Does this file exist: /home/shubham/models/fr-en/phrase-table.3.folder/Misc.dat If you do cat Misc.dat what does it say?
Hieu Hoang http://www.hoang.co.uk/hieu On 10 December 2016 at 11:30, Shubham Khandelwal <[email protected]> wrote: > Hi, > > Thanks Hieu for your reply. > I have used *CreateOnDiskPt* to binarize the model and stored in > *phrase-table.3.folder *using following command: > > > *~/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2 phrase-table.3.gz > phrase-table.3.folder* > > Also I have made changes in *moses.ini.3 (*i.e. I have converted > *PhraseDictionaryMemory* to *PhraseDictionaryCompact* as follows*) * > > PhraseDictionaryOnDisk name=TranslationModel0 num-features=4 > path=/home/shubham/models/fr-en/phrase-table.3.folder input-factor=0 > output-factor=0 > > Now, when I run it using :* ~/mosesdecoder/bin/moses -f moses.ini.3 * , > it gave following error after *Created input-output object*: > > *terminate called after throwing an instance of 'util::Exception'* > * what(): OnDiskPt/OnDiskWrapper.cpp:217 in uint64_t > OnDiskPt::OnDiskWrapper::GetMisc(const string&) const threw util::Exception > because `iter == m_miscInfo.end()'.* > *Couldn't find value for key NumSourceFactors* > *Aborted (core dumped)* > > Here, I do not know that what key value should I pass and how ? Can you > please help me in this regard. > > Thank you so much for your help. > > Regards, > Shubham > > On Fri, Dec 9, 2016 at 4:27 PM, Hieu Hoang <[email protected]> wrote: > >> This is a hierarchical model. You must binarize with CreateOnDiskPt for >> this model >> >> Hieu Hoang >> http://www.hoang.co.uk/hieu >> >> On 9 December 2016 at 08:18, Shubham Khandelwal <[email protected]> >> wrote: >> >>> Hello, >>> >>> Thanks. It worked now. I have created compact phrase table. >>> Now, when I am running it using following command: >>> >>> *~/mosesdecoder/bin/moses >>> -f ~/Translate/models/de-en/model/moses.ini.2 -threads all* >>> >>> Then, after creating input-output object, it gave following segmentation >>> fault error: >>> >>> Created input-output object : [14.796] seconds >>> Ich bin ein Student >>> Line 0: Initialize search took 0.000 seconds total >>> Translating: <s> Ich bin ein Student </s> ||| [0,0]=X (1) [0,1]=X (1) >>> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1) [1,2]=X (1) >>> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1) [2,4]=X (1) >>> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1) [4,5]=X (1) >>> [5,5]=X (1) >>> >>> Segmentation fault (core dumped) >>> >>> In my machine, I have 40GB RAM but still I am confused why it gave this >>> error. >>> Can you please help me in this regard. I have attached moses.ini.2 for >>> your reference. >>> >>> Thanks. >>> >>> Regards, >>> Shubham >>> >>> On Fri, Dec 9, 2016 at 2:02 AM, Hieu Hoang <[email protected]> wrote: >>> >>>> maybe try >>>> >>>> -encoding None >>>> >>>> On 08/12/2016 19:44, Shubham Khandelwal wrote: >>>> >>>> Hi Hieu, >>>> >>>> Thanks for your reply. >>>> Yes, I have used the absolute path and also I tried with -T but it did >>>> not work. >>>> Is there any other solution to this problem. >>>> >>>> Btw, Can anybody please upload the compact model of all pre-made models >>>> as this will take less space and also it will be very fast during decoding. >>>> >>>> Thanks. >>>> >>>> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <[email protected]> >>>> wrote: >>>> >>>>> the previous email you referred to says that the directory >>>>> >>>>> * binarised-model/ * >>>>> >>>>> *must exist before you run it, otherwise it will segfault. I would >>>>> also use absolute path to make sure, ie. not * >>>>> *binarised-model/phrase-table * >>>>> >>>>> *but * >>>>> >>>>> >>>>> * /home/shubham/moses/binarised-model/phrase-table * >>>>> >>>>> *The previous email exchange also says you should try to add the >>>>> argument * >>>>> >>>>> >>>>> * -T . * >>>>> >>>>> Hieu Hoang >>>>> http://www.hoang.co.uk/hieu >>>>> >>>>> On 8 December 2016 at 15:52, Shubham Khandelwal <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> This is just the reminder of my previous email. >>>>>> >>>>>> Thanking You. >>>>>> >>>>>> Regards, >>>>>> Shubham >>>>>> >>>>>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I have just downloaded phrase-table.2.gz (18GB) de-en model >>>>>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made >>>>>>> models. >>>>>>> Now, I am converting them to PhraseDictionaryCompact using following >>>>>>> command (for exmaple): >>>>>>> >>>>>>> >>>>>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in >>>>>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table >>>>>>> * >>>>>>> >>>>>>> But after passing 1/3, it gave following segementation fault error: >>>>>>> >>>>>>> *Pass 1/3: Creating hash function for rank assignment* >>>>>>> *Segmentation fault (core dumped)* >>>>>>> >>>>>>> I have found almost same issue on this thread: >>>>>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033 >>>>>>> However, I have provided the existing *binarised-model *folder in >>>>>>> the command. Also, I have the write-access in /tmp but still it >>>>>>> gave sementation fault. >>>>>>> >>>>>>> Can you please tell me what could be wrong here ? >>>>>>> >>>>>>> Thanking You. >>>>>>> >>>>>>> Regards, >>>>>>> Shubham >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>>> >>> >>> >> > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
