I have a wrapper for python moses and an additional python phrase based trainer as well (trainer is still under construction and uses a lot of memory currently).
The python wrapper simply calls the perl scripts as described on the statmt site through os.system. You are welcome to use it. Sanne -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: maandag 22 februari 2010 13:10 To: [email protected] Subject: Moses-support Digest, Vol 40, Issue 26 Send Moses-support mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit http://mailman.mit.edu/mailman/listinfo/moses-support or, via email, send a message with subject or body 'help' to [email protected] You can reach the person managing the list at [email protected] When replying, please edit your Subject line so it is more specific than "Re: Contents of Moses-support digest..." Today's Topics: 1. pymoses (christopher taylor) 2. 2.718 in the phrase-table (Calia) 3. Is reordering model a "must-be-used" component to use? (Calia) 4. skipping incompatible liboolm.a (Marce van Velden) 5. Re: skipping incompatible liboolm.a (Miles Osborne) 6. Re: problem in generating binary phrase table (Hieu Hoang) ---------------------------------------------------------------------- Message: 1 Date: Sun, 21 Feb 2010 22:10:26 -0400 From: christopher taylor <[email protected]> Subject: [Moses-support] pymoses To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1 is there a python wrapper for moses? ct ------------------------------ Message: 2 Date: Mon, 22 Feb 2010 13:26:12 +0900 (KST) From: Calia <[email protected]> Subject: [Moses-support] 2.718 in the phrase-table To: [email protected] Message-ID: <22598288.1266812772578.javamail.r...@twebmail> Content-Type: text/plain; charset="euc-kr" .Bold { font-weight: bold; } .Title { font-weight: bold; font-size: 18px; color: #cc3300; } .Code { border: #8b4513 1px solid; padding-right: 5px; padding-left: 5px;color: #000066; font-family: 'Courier New' , Monospace;background-color: #ff9933; } The exact purpose of the value 2.718 is what I am not sure of.As far as I know, the value is used for preferring the hypothesis with less phrases tothe one that use more phrases(words) for the same coverage of the source sentence duringthe prefix cost comparison inside priority queue( for the hyps that covers the same source range ).for example ) phrase_1 + phrase_2 + phrase_3 )to prefer hyp1 to hyp2 by multiplying hyp1 by 2.718.This is how I understand the use of the value, since longer phrase has empirically better translationthan the one made up of word-based translation.Is there any one who can confirm my belief, or to correct my conclusion?p.s. Is 2.718 the Euler's number? If it is, why is the weight determined as such? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/2010022 2/76dcc45f/attachment-0001.htm ------------------------------ Message: 3 Date: Mon, 22 Feb 2010 13:36:05 +0900 (KST) From: Calia <[email protected]> Subject: [Moses-support] Is reordering model a "must-be-used" component to use? To: [email protected] Message-ID: <2831709.1266813365495.javamail.r...@twebmail> Content-Type: text/plain; charset="euc-kr" .Bold { font-weight: bold; } .Title { font-weight: bold; font-size: 18px; color: #cc3300; } .Code { border: #8b4513 1px solid; padding-right: 5px; padding-left: 5px;color: #000066; font-family: 'Courier New' , Monospace;background-color: #ff9933; } I wonder if I may exclude the reordering model during search.Since I came up with my morpho-syntactical preprocessor to transform source languagein both training-time and run-time, and reordering model deals with the local reordering or wordsduring translation, there doesn't seem to be the need for using it if the preprocessorcompletely got rid of local/global distortion, even for language model?Is my hypothesis justified?Actually, from my subjective evaluation, using phrase-table and lm alone shows better result thanwhen I use them with reordering model. But I am not sure of my theory. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/2010022 2/22541254/attachment-0001.htm ------------------------------ Message: 4 Date: Mon, 22 Feb 2010 12:40:51 +0100 From: Marce van Velden <[email protected]> Subject: [Moses-support] skipping incompatible liboolm.a To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" Hi, I get the folowing error when trying to compile moses on a intel64 pc. What could cause the liboolm.a to be incompatible? (/usr/bin/ld: skipping incompatible /home/marce/srilm64/lib/i686/liboolm.a when searching for -loolm) ma...@moses:~/moses/trunk$ sudo make make all-recursive make[1]: Entering directory `/home/marce/moses/trunk' Making all in moses/src make[2]: Entering directory `/home/marce/moses/trunk/moses/src' make all-am make[3]: Entering directory `/home/marce/moses/trunk/moses/src' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/home/marce/moses/trunk/moses/src' make[2]: Leaving directory `/home/marce/moses/trunk/moses/src' Making all in moses-cmd/src make[2]: Entering directory `/home/marce/moses/trunk/moses-cmd/src' g++ -g -O2 -L/home/marce/srilm64/lib/i686 -o moses Main.o mbr.o IOWrapper.o TranslationAnalysis.o LatticeMBR.o -L../../moses/src -lmoses -L/usr/include/boost/lib -lboost_thread-mt -loolm -ldstruct -lmisc -lz /usr/bin/ld: skipping incompatible /home/marce/srilm64/lib/i686/liboolm.a when searching for -loolm /usr/bin/ld: cannot find -loolm collect2: ld returned 1 exit status make[2]: *** [moses] Error 1 make[2]: Leaving directory `/home/marce/moses/trunk/moses-cmd/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/marce/moses/trunk' make: *** [all] Error 2 Thanks, Marce -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/2010022 2/d05fb65d/attachment-0001.htm ------------------------------ Message: 5 Date: Mon, 22 Feb 2010 11:56:27 +0000 From: Miles Osborne <[email protected]> Subject: Re: [Moses-support] skipping incompatible liboolm.a To: Marce van Velden <[email protected]> Cc: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1 this is a standard error. you need to build SRILM using 64-bit support (i686-m64) Miles On 22 February 2010 11:40, Marce van Velden <[email protected]> wrote: > Hi, > I get the folowing error when trying to compile moses on a intel64 pc. > What could cause the liboolm.a to be incompatible? > (/usr/bin/ld: skipping incompatible > /home/marce/srilm64/lib/i686/liboolm.a > when searching for -loolm) > ma...@moses:~/moses/trunk$ sudo make > make ?all-recursive > make[1]: Entering directory `/home/marce/moses/trunk' > Making all in moses/src > make[2]: Entering directory `/home/marce/moses/trunk/moses/src' > make ?all-am > make[3]: Entering directory `/home/marce/moses/trunk/moses/src' > make[3]: Nothing to be done for `all-am'. > make[3]: Leaving directory `/home/marce/moses/trunk/moses/src' > make[2]: Leaving directory `/home/marce/moses/trunk/moses/src' > Making all in moses-cmd/src > make[2]: Entering directory `/home/marce/moses/trunk/moses-cmd/src' > g++ ?-g -O2 ?-L/home/marce/srilm64/lib/i686 -o moses Main.o mbr.o > IOWrapper.o TranslationAnalysis.o LatticeMBR.o -L../../moses/src > -lmoses -L/usr/include/boost/lib -lboost_thread-mt -loolm -ldstruct > -lmisc -lz > /usr/bin/ld: skipping incompatible > /home/marce/srilm64/lib/i686/liboolm.a > when searching for -loolm > /usr/bin/ld: cannot find -loolm > collect2: ld returned 1 exit status > make[2]: *** [moses] Error 1 > make[2]: Leaving directory `/home/marce/moses/trunk/moses-cmd/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/home/marce/moses/trunk' > make: *** [all] Error 2 > Thanks, > Marce > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ------------------------------ Message: 6 Date: Mon, 22 Feb 2010 12:10:19 +0000 From: Hieu Hoang <[email protected]> Subject: Re: [Moses-support] problem in generating binary phrase table To: Zahurul Islam <[email protected]>, "[email protected]" <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset="utf-8" hi zahurul your phrase table doesn't appear to be sorted. When i run grep -n "^0 ( 0,0 % ) 0 " phrase-table3000 the lines with the source you meantioned 0 ( 0,0 % ) 0 appears on line 538, 540, 550, 774 the phrase table needs to be sorted before binarizing. To sort the file, run export LC_ALL=C sort phrase-table3000 > phrase-table3000.sorted On 22/02/2010 09:54, Zahurul Islam wrote: > Hi, > My phrase table is very big (15 GB without compression). I was > confused to follow your instruction. I took first 3000 line from the > phrase table and tried to build binary phrase table. I met that error > again. I deleted line:386. Again I met same kind of error at line 539: > > line(539): '0 ( 0,0 % ) 0 ||| 0 ( 0.0 % ) 0 ||| (0) (1) (2) (3) (4) > (5) ||| (0) (1) (2) (3) (4) (5) ||| 0.84 0.248297 0.933333 0.136567 2.718' > f: 6 13 15 5 3 7 13 > Abort > > I have attached the experimental portion of phrase table with this > email. I hope it will help us to find out the problem. Thank you very > much for help. > > Regards, > Zahurul > > > On Fri, Feb 19, 2010 at 11:31 PM, Hieu Hoang <[email protected] > <mailto:[email protected]>> wrote: > > can you check if there's 2 lines that have the same source? if so, > can you send me the 2 lines > > > On 19/02/2010 16:21, Zahurul Islam wrote: >> Hi, >> I was trying to generate binary phrase table using following >> command: ("LC_ALL=C" was set) >> $zcat phrase-table.gz | sort | /X/moses/misc/processPhraseTable >> -ttable 0 0 - -nscores 5 -out phrase-table >> >> I met following error: >> ERROR: xsource phrase already inserted (B)! >> line(386): '000 000 ||| 000 000 other ||| (0) (1) ||| (0) (1) () >> ||| 1 0.219318 0.0194175 0.000347081 2.718' >> f: 2 6 6 >> Abort >> >> Any idea what's going wrong here? >> Thanks in advance. >> >> Regards, >> Zahurul Islam >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] <mailto:[email protected]> >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/2010022 2/a856f52e/attachment.htm ------------------------------ _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support End of Moses-support Digest, Vol 40, Issue 26 ********************************************* _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
