Hi Ivan,
You could fix it by looking at the file
src/moses/scripts/training/phrase-extract/score.cpp with method void
LexicalTable::load( char *fileName ) inside:
vector<string> token = tokenize( line );
if (token.size() != 3) {
cerr << "line " << i << " in " << fileName << " has wrong number of
tokens, skipping:\n" <<
token.size() << " " << token[0] << " " << line << endl;
continue;
}
You could either modfiy the method tokenize, or relax the if condition.
Hope that helps,
Cheers.
Thang
On Mon, Nov 2, 2009 at 8:29 PM, Ivan Uemlianin <[email protected]>wrote:
> Dear All
>
> I have Moses running fine on MacOSX. Now I am setting it up on Windows
> using Cygwin.
>
> The current error I'm working on is that the file model/lex.f2e
> occasionally has a space as its first field. Does anyone know how this
> comes about and/or how I can fix it?
>
> Some details:
>
> I'm running the simple train-factored-phrase-model.perl scripts from the
> step through page, like this:
>
>
> cmd = nohup nice \
> /full/path/to/train-factored-phrase-model.perl \
> -scripts-root-dir \
> /full/path/to/scripts-20091102-1102 \
> -root-dir \
> /full/path/to/tf \
> -corpus /full/path/to/tf/corpus/projname.tok \
> -f cy \
> -e en \
> -alignment grow-diag-final-and \
> -reordering msd-bidirectional-fe \
> -lm 0:3:/full/path/to/tf/lm_irst/projname.en.irstlm.gz:1
>
>
> Everything seems to run OK --- I mean it doesn't crash or freeze --- but
> the translator doesn't work. stderr from the script has the following
> warnings:
>
>
> Loading lexical translation table from
> /home/ivan/moses_tools/factory/tf/model/lex.f2e
> line 34 in /home/ivan/moses_tools/factory/tf/model/lex.f2e has wrong
> number of tokens, skipping:
> 2 gwyntoedd gwyntoedd 0.0087719
> line 83 in /home/ivan/moses_tools/factory/tf/model/lex.f2e has wrong
> number of tokens, skipping:
> 2 droi droi 0.4000000
>
>
> The relevant lines in lex.f2e have a space as their first token, as in:
>
>
> the gwyntoedd 0.0225564
> gwyntoedd 0.0150376
> a gwyntoedd 0.0075188
>
>
> Any help would be much appreciated. Once it's all working I'll post
> full guidance on getting Moses running under Cygwin.
>
> Best wishes
>
> Ivan
>
>
> --
> ********************************
> Ivan Uemlianin
>
> Canolfan Bedwyr
> Safle'r Normal Site
> Prifysgol Bangor University
> BANGOR
> Gwynedd
> LL57 2PZ
>
> [email protected]
> ********************************
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Luong Minh Thang
WING group, School of Computing, National University of Singapore
http://wing.comp.nus.edu.sg/~lmthang
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
