Thanks All...I have been able to move forward to "EVALUATION" finally.
And I am still facing some problems
when I run the following code given in the step-by-step guide
I am facing the following errors
1.(3) Preparing data for training recasing model @ Mon Nov 24 23:27:34 IST
2008
recaser/aligned.lowercased
utf8 "\xF3" does not map to Unicode at
bin/moses-scripts/scripts-20080912-1904/r
ecaser/train-recaser.perl line 79, <CORPUS> line 14.
Malformed UTF-8 character (unexpected non-continuation byte 0x6e,
immediately af
ter start byte 0xf3) in lc at
bin/moses-scripts/scripts-20080912-1904/recaser/tr
ain-recaser.perl line 73, <CORPUS> line 14.
utf8 "\xE9" does not map to Unicode at
bin/moses-scripts/scripts-20080912-1904/r
ecaser/train-recaser.perl line 79, <CORPUS> line 54.
And Many Other Lines from the corpus.When I went and examined those
particular lines it seemed to me that there were Non-English Characters
there,which is expected as I am doing this with English-Deutsch. Changing
$LANG to LANG=en_US.ISO8859-1
Didnt seem to make any difference .(Or was I doing it wrong?).The when I
looked into train-recaser.perl code
open(CORPUS,$CORPUS);
binmode(CORPUS, ":utf8");
Is this line causing Problems ? Or is it ok ? Or is this data not in UTF-8
and nothing can be done ? A earlier thread said that
2.Use of uninitialized value $SCRIPTS_ROOTDIR in concatenation (.) or string
at /PATH-TO-BIN/bin/moses-scripts/s
cripts-20080912-1904/training/train-factored-phrase-model.perl line 105.
On Sun, Nov 16, 2008 at 3:14 AM, Radek Bartoň <[EMAIL PROTECTED]>wrote:
> On Saturday 15 of November 2008 22:05:49 [EMAIL PROTECTED] wrote:
> > On Sat, Nov 15, 2008 at 20:58:58 +0000, Eric Kow wrote:
> > > But for some reason, this doesn't work for me and the SCRIPTS_ROOTDIR
> > > is silently set to the empty string. I ended up hard-coding
> > > SCRIPTS_ROOTDIR in the mert-moses scripts (and for that matter, in
> > > train-factored-phrase-model), but does anybody have a better solution?
> >
> > Whoops... one other thing I had to do was to tell Perl to set the
> > environment variable too:
> >
> > if (defined($ENV{"SCRIPTS_ROOTDIR"})) {
> > $SCRIPTS_ROOTDIR = $ENV{"SCRIPTS_ROOTDIR"};
> > } else {
> > $ENV{"SCRIPTS_ROOTDIR"} = $SCRIPTS_ROOTDIR;
> > }
> >
> > Which seems to work, although I'd definitely be interested in learning
> > the right answer :-)
>
> I had the same issue recently. I've fixed it by defining SCRIPTS_ROOTDIR
> variable in the environment before mert-moses.pl script execution.
>
> --
> Ing. Radek Bartoň
>
> Faculty of Information Technology
> Department of Computer Graphics and Multimedia
> Brno University of Technology
>
> E-mail: [EMAIL PROTECTED]
> Web: http://blackhex.no-ip.org
> Jabber: [EMAIL PROTECTED]
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
