Hi Asim - I got a similar result (phrase-table of 20 bytes) when I got the -f and -e arguments the wrong way round for train-model.perl
eg nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train -corpus ~/corpus/ees.en-id.clean -f en -e id ... but it worked ok with nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train -corpus ~/corpus/ees.en-id.clean -f id -e en ... ------------------------------- Julian Myerscough [email protected] wrote: > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. Re: Empty phrase-table extract.inv.sorted extract.sorted > (Aasim Ali) > 2. Re: Empty phrase-table extract.inv.sorted extract.sorted > (Barry Haddow) > 3. mosesserver weird behaviour (Montse Cuadros) > 4. Re: mosesserver weird behaviour (Barry Haddow) > 5. Re: mosesserver weird behaviour (Marwen AZOUZI) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 17 Jul 2012 22:31:08 -0700 (PDT) > From: Aasim Ali <[email protected]> > Subject: Re: [Moses-support] Empty phrase-table extract.inv.sorted > extract.sorted > To: Moses SMT toolkit support mailing list <[email protected]> > Message-ID: > <[email protected]> > Content-Type: text/plain; charset="utf-8" > > It is giving the same result (empty phrase-table, extract.sorted, and > extract.inv.sorted) for news-commentart-v7.fr-en.clean corpora. > > > > ________________________________ > From: Aasim Ali <[email protected]> > To: Moses SMT toolkit support mailing list <[email protected]> > Sent: Tuesday, 17 July 2012, 17:56 > Subject: Empty phrase-table extract.inv.sorted extract.sorted > > > Hi > > I am running train-model.perl. At the end the script outputs that moses.ini > file has been written. But, the following files in the training/model folder > remain of 20B each, and the extracted files carry 0B size: > phrase-table.gz > extract.sorted.gz > extract.inv.sorted.gz > > I have tried to look into the mailing list, and have checked suggested > things, e.g. the files output by Giza++ seem fine to me. The output (STDOUT) > and error (STDERR) are attached herewith, for investigation of problem. > > Thanks and regards, > Aasim Ali > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20120717/ae5c55db/attachment-0001.htm > > ------------------------------ > > Message: 2 > Date: Wed, 18 Jul 2012 09:26:58 +0100 > From: Barry Haddow <[email protected]> > Subject: Re: [Moses-support] Empty phrase-table extract.inv.sorted > extract.sorted > To: [email protected], Aasim Ali <[email protected]> > Message-ID: <[email protected]> > Content-Type: Text/Plain; charset="utf-8" > > Hi Aasim > > I could not find your earlier mail, and didn't see the attachments. > > The problem is that there is an error somewhere in the training pipeline, and > since extract was multi-threaded the errors are not caught at the time they > occur. The only way to track down the error is to study the log files > carefully, and note that GIZA++ emits lots of spurious warnings. > > cheers - Barry > > On Wednesday 18 Jul 2012 06:31:08 Aasim Ali wrote: >> It is giving the same result (empty phrase-table, extract.sorted, and >> extract.inv.sorted) for news-commentart-v7.fr-en.clean corpora. >> >> >> >> ________________________________ >> From: Aasim Ali <[email protected]> >> To: Moses SMT toolkit support mailing list <[email protected]> >> Sent: Tuesday, 17 July 2012, 17:56 >> Subject: Empty phrase-table extract.inv.sorted extract.sorted >> >> >> Hi >> >> I am running train-model.perl. At the end the script outputs that moses.ini >> file has been written. But, the following files in the training/model >> folder remain of 20B each, and the extracted files carry 0B size: >> phrase-table.gz >> extract.sorted.gz >> extract.inv.sorted.gz >> >> I have tried to look into the mailing list, and have checked suggested >> things, e.g. the files output by Giza++ seem fine to me. The output >> (STDOUT) and error (STDERR) are attached herewith, for investigation of >> problem. >> >> Thanks and regards, >> Aasim Ali >> > > > ------------------------------ > > Message: 3 > Date: Wed, 18 Jul 2012 14:08:17 +0200 > From: Montse Cuadros <[email protected]> > Subject: [Moses-support] mosesserver weird behaviour > To: [email protected] > Message-ID: > <CAAUZdmjRf0CyEFbDqRmo=c1gdKsPb=f3bokpsny878sewq6...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Dear all, > > We have been working with moses for translating some files, and recasing > them acording to a prebuilt models trained in moses. > > We don't have problems when using moses in both cases. > > Furthermore, we want to use the same models with mosesserver. Here the > problem starts. > > We have compiled moses and mosesserver together with irstml, srlim and > rpc_xml and everything worked fine. > > The problems have started when translating and recasing files from spanish > and french that have "?","?", ... ... and these kind of letters. > With moses we don't have problems with encodings and with mosesserver yes, > having them compiled together, with the same parameters, and with the same > configuration files. > > Any clue of what is happening to us? > > We will appreciate any suggestion. > > Best, > Mon > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20120718/df8cea94/attachment-0001.htm > > ------------------------------ > > Message: 4 > Date: Wed, 18 Jul 2012 13:29:34 +0100 > From: Barry Haddow <[email protected]> > Subject: Re: [Moses-support] mosesserver weird behaviour > To: [email protected] > Cc: Montse Cuadros <[email protected]> > Message-ID: <[email protected]> > Content-Type: Text/Plain; charset="utf-8" > > Hi Mon, > > When using mosesserver, the clients have to encode the characters correctly > for transmission in xml-rpc. You may have to make some additional libraries > calls in the client to get the encoding correct. The supplied sample clients > should work correctly, but I have seen problems in the past with (eg) buggy > perl libraries. > > To check what messages are being passed, you should set the XMLRPC_TRACE_XML > environment variable to 1 before launching the mosesserver. You could also > run > mosesserver with -v 1 to monitor what it is receiving/sending. This should > tell you where the character encoding is being corrupted. > > cheers - Barry > > > On Wednesday 18 Jul 2012 13:08:17 Montse Cuadros wrote: >> Dear all, >> >> We have been working with moses for translating some files, and recasing >> them acording to a prebuilt models trained in moses. >> >> We don't have problems when using moses in both cases. >> >> Furthermore, we want to use the same models with mosesserver. Here the >> problem starts. >> >> We have compiled moses and mosesserver together with irstml, srlim and >> rpc_xml and everything worked fine. >> >> The problems have started when translating and recasing files from spanish >> and french that have "?","?", ... ... and these kind of letters. >> With moses we don't have problems with encodings and with mosesserver yes, >> having them compiled together, with the same parameters, and with the same >> configuration files. >> >> Any clue of what is happening to us? >> >> We will appreciate any suggestion. >> >> Best, >> Mon >> > > > > ------------------------------ > > Message: 5 > Date: Wed, 18 Jul 2012 15:22:17 +0200 > From: Marwen AZOUZI <[email protected]> > Subject: Re: [Moses-support] mosesserver weird behaviour > To: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8" > > Hi Montse, > > I encountered the same problem with the provided Perl client > (https://github.com/moses-smt/mosesdecoder/blob/master/contrib/server/client.perl). > I fixed it by using the attached Perl client (it uses a different library). > > Hope it will work for you. > > Marwen > > Le 18/07/2012 14:29, Barry Haddow a ?crit : >> Hi Mon, >> >> When using mosesserver, the clients have to encode the characters correctly >> for transmission in xml-rpc. You may have to make some additional libraries >> calls in the client to get the encoding correct. The supplied sample clients >> should work correctly, but I have seen problems in the past with (eg) buggy >> perl libraries. >> >> To check what messages are being passed, you should set the XMLRPC_TRACE_XML >> environment variable to 1 before launching the mosesserver. You could also >> run >> mosesserver with -v 1 to monitor what it is receiving/sending. This should >> tell you where the character encoding is being corrupted. >> >> cheers - Barry >> >> >> On Wednesday 18 Jul 2012 13:08:17 Montse Cuadros wrote: >>> Dear all, >>> >>> We have been working with moses for translating some files, and recasing >>> them acording to a prebuilt models trained in moses. >>> >>> We don't have problems when using moses in both cases. >>> >>> Furthermore, we want to use the same models with mosesserver. Here the >>> problem starts. >>> >>> We have compiled moses and mosesserver together with irstml, srlim and >>> rpc_xml and everything worked fine. >>> >>> The problems have started when translating and recasing files from spanish >>> and french that have "?","?", ... ... and these kind of letters. >>> With moses we don't have problems with encodings and with mosesserver yes, >>> having them compiled together, with the same parameters, and with the same >>> configuration files. >>> >>> Any clue of what is happening to us? >>> >>> We will appreciate any suggestion. >>> >>> Best, >>> Mon >>> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > -------------- next part -------------- > #!/usr/bin/perl > # Perl sample client for mosesserver (Created by Marwen AZOUZI) > > use strict; > #use Data::Dumper; > use RPC::XML; > use RPC::XML::Client; > use Encode; > > my $text = $ARGV[0]; > > my $client = RPC::XML::Client->new('http://127.0.0.1:8000/RPC2'); > $RPC::XML::ENCODING = "UTF-8"; > my $request = > RPC::XML::request->new('translate',RPC::XML::struct->new({'text' => > RPC::XML::string->new($text)})); > my $response = $client->send_request($request); > > if (!$response) { > print $RPC::XML::ERROR, "\n"; > } else { > print Encode::encode("utf-8",$response->{text}->value); > } > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 69, Issue 52 > ********************************************* _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
