Hi,
The error message said
/disque2/Preparation/syntactic/hierarchical_PSCT/rules-table
is a directory. But IIRC phrase table 6 is "SCFG" and expects a file.
If you have binarized the phrase table specify a different magic number
in the moses.ini.
Nothing to do with the language model but, for the record, SRILM does
have a binary format. Just not a memory mapped binary format.
Kenneth
On 03/22/14 04:24, Massinissa Ahmim wrote:
> Hi Hieu,
>
> It's actually KenLM I've just updated my moses.ini changing the 0 to 8
> but I'm still getting the same error
>
> Regards
>
> Massinissa
>
>
> 2014-03-22 11:40 GMT+01:00 Hieu Hoang <[email protected]
> <mailto:[email protected]>>:
>
> did you binarize the LM and what did you binarize it with? KenLM or
> IRSTLM?
>
> at the moment, it's set to SRILM, which doesn't have a binary format
>
>
>
> On 22 March 2014 10:10, Massinissa Ahmim
> <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Hieu,
>
> I used the same binarisation command as the one stated here-above
>
> I get this error when I try to translate something like : echo
> 'ceci est un test' | /mosesdecoder/bin/moses_chart -f moses.ini
>
> Here is my moses.ini :
>
> #########################
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
> 1 T 1
>
> # translation tables: table type (hierarchical(0), textual (0),
> binary (1)), source-factors, target-factors, number of scores, file
> # OLD FORMAT is still handled for back-compatibility
> # OLD FORMAT translation tables: source-factors, target-factors,
> number of scores, file
> # OLD FORMAT a binary table type (1) is assumed
> [ttable-file]
> 6 0 0 1 /disque2/Preparation/syntactic/hierarchical_PSCT/rules-table
> 6 0 0 1
> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>
> # no generation models, no generation-file section
>
> # language models: type(srilm/irstlm), factors, order, file
> [lmodel-file]
> 0 0 5 /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
>
>
> # limit on how many phrase translations e for each phrase f are
> loaded
> # 0 = all elements loaded
> [ttable-limit]
> 20
>
>
> # language model weights
> [weight-l]
> 0.5000
>
>
> # translation model weights
> [weight-t]
> 0.20
> 0.20
> 0.20
> 0.20
> 0.20
> 1.0
>
> # no generation models, no weight-generation section
>
> # word penalty
> [weight-w]
> -1
>
> [cube-pruning-pop-limit]
> 1000
>
> [non-terminals]
> X
>
> [search-algorithm]
> 3
>
> [inputtype]
> 3
>
> [max-chart-span]
> 20
> 1000
>
>
>
> 2014-03-21 20:49 GMT+01:00 Hieu Hoang <[email protected]
> <mailto:[email protected]>>:
>
> i've never seen that error before. Did you change anything
> else, other than rebinarizing the phrase-table? What is the
> exact command you ran when you got this error? Can I have a
> look at your moses.ini file
>
>
> On 21 March 2014 18:32, Massinissa Ahmim
> <[email protected]
> <mailto:[email protected]>> wrote:
>
> Thank you Hieu,
>
> I've binarised my rule-table again and it seems to be
> successfully loaded but I'm getting this message now :
>
> Reading
> /disque2/Preparation/syntactic/hierarchical_PSCT/rules-table
>
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
>
> ****************************************************************************************************
> Exception: <fd:3>: Is a directory from zlib
>
> What can I do?
>
> Thanks !
>
> Regards
>
> Massinissa
>
>
> 2014-03-21 11:18 GMT+01:00 Hieu Hoang
> <[email protected] <mailto:[email protected]>>:
>
> everything seems to be ok. Are you sure the server
> you are running on has access to the directory
>
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>
> Also, you should consider updating your version of
> Moses to the current github version, or version 2.1.1
>
>
> On 20 March 2014 11:20, Massinissa Ahmim
> <[email protected]
> <mailto:[email protected]>> wrote:
>
> Dear all,
>
> I've managed to train a hierarchical model using
> the following command :
>
> nohup
> /mosesdecoder/scripts/training/train-model.perl
> --hierarchical --glue-grammar
> --score-options="--GoodTuring" -root-dir
> /disque2/Preparation/syntactic/hierarchical_PSCT
> -corpus
> /disque2/Preparation/backoff/PSCT.tok.uni.low
> -f fr -e en -lm
>
> 0:5:/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
> -external-bin-dir /root/external-bin-dir/ -mgiza
> -mgiza-cpus 30 >& training.out &
>
> The resulting model works great but it is very
> slow.
>
> So I used CreatOnDiskPt to binarise the rule
> table as follows :
>
> /home/Moses/mosesdecoder/bin/CreateOnDiskPt 1 1
> 5 20 2 model/rule-table.gz rules-table
>
> This outputs the following files
>
> -rw-r--r-- 1 root root 85 20 mars 11:35
> Misc.dat
> -rw-r--r-- 1 root root 781509322 20 mars 11:35
> Source.dat
> -rw-r--r-- 1 root root 1758588137 20 mars 11:35
> TargetColl.dat
> -rw-r--r-- 1 root root 1555353459 20 mars 11:35
> TargetInd.dat
> -rw-r--r-- 1 root root 400714 20 mars 11:35
> Vocab.dat
>
> and I've updated my moses.ini as follows :
>
> 6 0 0 1
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
> 6 0 0 1
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>
> But as I try to use it, I'm getting this :
>
> Defined parameters (per moses.ini or switch):
> config: moses.ini
> cube-pruning-pop-limit: 1000
> input-factors: 0
> inputtype: 3
> lmodel-file: 0 0 5
>
> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
> mapping: 0 T 0 1 T 1
> max-chart-span: 20 1000
> non-terminals: X
> search-algorithm: 3
> ttable-file: 6 0 0 1
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
> 6 0 0 1
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
> ttable-limit: 20
> weight-l: 0.5000
> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
> weight-w: -1
> /mosesdecoder/bin
> ScoreProducer: WordPenalty start: 0 end: 1
> ScoreProducer: !UnknownWordPenalty start: 1 end: 2
> Loading lexical distortion models...have 0 models
> Start loading LanguageModel
>
> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
> : [0.000] seconds
>
> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm:
> line 80492: reached EOF before \end\
> ScoreProducer: LM start: 2 end: 3
> Finished loading LanguageModels : [0.056] seconds
> Using uniform ttable-limit of 20 for all
> translation tables.
> Start loading PhraseTable
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
> : [0.056] seconds
> filePath:
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
> ScoreProducer: PhraseModel start: 3 end: 4
> Start loading PhraseTable
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
> : [0.056] seconds
> filePath:
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
> ScoreProducer: PhraseModel:2 start: 4 end: 5
> Finished loading phrase tables : [0.056] seconds
> max-chart-span: 20
> max-chart-span: 1000
> Start loading phrase table from
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
> : [0.056] seconds
> Can't read
>
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>
> So I wondering if I did something wrong with my
> training command/binarisation or with the
> parameters in the moses.ini
>
>
> Many thanks
>
> Regads
>
> MA
>
>
>
>
> --
>
> Description : Description :
> lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44
> 40 84*
>
> *Email :**
> **[email protected]
> <mailto:[email protected]>***
>
> *Website :** **www.linguacustodia.com
> <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com>*
>
> ü Pensez à l’environnement, n’imprimez ce
> courriel que si nécessaire.
>
> Please do not print this email unless it is
> absolutely necessary. Spread environmental
> awareness.
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected] <mailto:[email protected]>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
>
> --
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :** **[email protected]
> <mailto:[email protected]>***
>
> *Website :** **www.linguacustodia.com
> <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com>*
>
> ü Pensez à l’environnement, n’imprimez ce courriel que
> si nécessaire.
>
> Please do not print this email unless it is absolutely
> necessary. Spread environmental awareness.
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
>
> --
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :** **[email protected]
> <mailto:[email protected]>***
>
> *Website :** **www.linguacustodia.com
> <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com>*
>
> ü Pensez à l’environnement, n’imprimez ce courriel que si
> nécessaire.
>
> Please do not print this email unless it is absolutely
> necessary. Spread environmental awareness.
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
>
> --
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :** **[email protected]
> <mailto:[email protected]>***
>
> *Website :** **www.linguacustodia.com
> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com>*
>
> ü Pensez à l’environnement, n’imprimez ce courriel que si nécessaire.
>
> Please do not print this email unless it is absolutely necessary. Spread
> environmental awareness.
>
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
