To be honest, it may be best to use the original binary file format that you get when you execute processPhraseTable processLexicalTable There are existing tools to query this phrase-table format without the decoder queryPhraseTable queryLexicalTable
For hierarchical models, you can use the phrase-table created by CreateOnDiskPt You can query it without the decoder using queryOnDiskPt On 12 December 2014 at 10:48, Raj Dabre <[email protected]> wrote: > > Hey, > > This is actually quite close to what I need. > There is one limitation though.... this works only for phrase tables and > not for reordering tables. > I can do a workround by treating the reordering table as a phrase > table.... but this code is designed to accept the only phrase table format. > > So I decided to change the code a little. > More specifically: I tried to append a dummy alignment and count string > component like "0-0 ||| 1 3 1 ||| |||" > This makes the line resemble the standard phrase table format. > > But I get errors like: std::bad_alloc > Since I am using hardcoded strings... this might be the issue. > Kindly advise me on how I can do this. > > Regards. > > > On Thu, Dec 11, 2014 at 8:17 PM, Hieu Hoang <[email protected]> wrote: >> >> there's also is the 'KenPT' that was create by Nikolay Bogoychev. We've >> been meaning to see if it could be incorporated into the KenLM main >> repository >> https://github.com/XapaJIaMnu >> >> On 10 December 2014 at 20:27, Kenneth Heafield <[email protected]> >> wrote: >> >>> I for one would like a version of phrase table querying that doesn't >>> depend on the rest of Moses. For cleanliness and for portability >>> (there's no reason this shouldn't kill cdec's phrase table for example). >>> >>> Kenneth >>> >>> On 12/10/14 13:39, Marcin Junczys-Dowmunt wrote: >>> > I am on it, sort of. I am waiting for Raj, to give me a stack trace. I >>> > admit this code is basically abandoned, but let's see what we can do. >>> > >>> > W dniu 10.12.2014 o 19:35, Hieu Hoang pisze: >>> >> I see. I'm not sure why the querying depends on StaticData. There may >>> >> be some weights is wants to read. Only Marcin will know about that. >>> >> You could try to debug yourself. >>> >> >>> >> If you intend to develop with Moses, I would urge you to update to the >>> >> master branch of Moses. People forget how old code works and often >>> >> can't answer your questions. The new code has also been cleaned up and >>> >> rationalised. For instance, the 'translation system' functionality was >>> >> problematic and has been deleted >>> >> >>> >> On 10 December 2014 at 15:53, Raj Dabre <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >> >>> >> I am not using any moses.ini because I am not translating >>> anything. >>> >> My focus is on reading the phrase tables using a querying program >>> >> written by Marcin. >>> >> >>> >> On Thu, Dec 11, 2014 at 12:50 AM, Hieu Hoang <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >> >>> >> Are you using the translation system functionality in v1? That >>> >> seems to be the problem. Can I have a look @ the moses.ini. >>> >> >>> >> On 10 December 2014 at 15:44, Raj Dabre <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >> >>> >> I am using an older version (V1). >>> >> I am not running a translation experiment but using Java >>> >> Native Interface to call C functions. >>> >> Can you tell me what might be the problem ? >>> >> I can surely say that the root of the problem is in the >>> >> StaticData.cpp file. >>> >> It seems that I am creating 2 instances of a class trying >>> >> to access a background moses process. >>> >> Does this sound about right ? >>> >> >>> >> >>> >> On Thu, Dec 11, 2014 at 12:37 AM, Hieu Hoang >>> >> <[email protected] <mailto:[email protected]>> wrote: >>> >> >>> >> what version of Moses are you using? The current >>> >> master branch on github, or an older version (version >>> >> 1 etc)? >>> >> >>> >> Can I please see the moses.ini file you're using too >>> >> >>> >> On 10 December 2014 at 12:34, Raj Dabre >>> >> <[email protected] <mailto:[email protected]>> >>> wrote: >>> >> >>> >> Dear all, >>> >> >>> >> I am using a JNI interface to the CPP code that >>> >> reads from binary phrase tables. >>> >> >>> >> I get an error like: Mismatch between number of >>> >> word penalties and number of translation systems >>> >> >>> >> This happens when I create 2 objects for reading >>> >> from 2 different tables. >>> >> >>> >> My understanding is that there is some problem >>> >> with Static data but I am not sure. >>> >> >>> >> Can anyone help me ? >>> >> >>> >> -- >>> >> Raj Dabre. >>> >> Research Student, >>> >> Graduate School of Informatics, >>> >> Kyoto University. >>> >> CSE MTech, IITB., 2011-2014 >>> >> >>> >> >>> >> _______________________________________________ >>> >> Moses-support mailing list >>> >> [email protected] <mailto: >>> [email protected]> >>> >> >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> Hieu Hoang >>> >> Research Associate >>> >> University of Edinburgh >>> >> http://www.hoang.co.uk/hieu >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> Raj Dabre. >>> >> Research Student, >>> >> Graduate School of Informatics, >>> >> Kyoto University. >>> >> CSE MTech, IITB., 2011-2014 >>> >> >>> >> >>> >> _______________________________________________ >>> >> Moses-support mailing list >>> >> [email protected] <mailto:[email protected]> >>> >> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> Hieu Hoang >>> >> Research Associate >>> >> University of Edinburgh >>> >> http://www.hoang.co.uk/hieu >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> Raj Dabre. >>> >> Research Student, >>> >> Graduate School of Informatics, >>> >> Kyoto University. >>> >> CSE MTech, IITB., 2011-2014 >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> Hieu Hoang >>> >> Research Associate >>> >> University of Edinburgh >>> >> http://www.hoang.co.uk/hieu >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> Moses-support mailing list >>> >> [email protected] >>> >> http://mailman.mit.edu/mailman/listinfo/moses-support >>> > >>> > >>> > >>> > _______________________________________________ >>> > Moses-support mailing list >>> > [email protected] >>> > http://mailman.mit.edu/mailman/listinfo/moses-support >>> > >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >> >> >> -- >> Hieu Hoang >> Research Associate >> University of Edinburgh >> http://www.hoang.co.uk/hieu >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > > -- > Raj Dabre. > Research Student, > Graduate School of Informatics, > Kyoto University. > CSE MTech, IITB., 2011-2014 > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
