You're right, there is a difference is model scores and output when using the binary or memory phrase-table.
This happens when I'm using Moses v1, but not if I'm using the current Moses in github. Your lexical reordering models are fairly strange because they depend on a source factor that only 1 phrase-table has. So you want to use only 1 lex. reordering model for each phrase-table? I'm not sure if Moses is working as you intend. I'm not sure what it's doing either! However, I think it's an interesting experiment so if you want to pursue this and you're willing to do some coding, I can help you implement this. On 12 February 2014 14:28, Mahmoud Ghoneim <[email protected]> wrote: > Thanks Hieu for your reply. > I changed my email address and I needed to register the new one. I did. > > I ran it on my machines without LMs and without all the command line > arguments and again I got two different outputs! > The attached file contains the two outputs. > > Also if "both lexical reordering models should be used whenever a > hypothesis is created, by either translation model", then the text-based > model is not working as supposed. Would you please review the verbose files > I sent with the previous email. (I mentioned the exact lines that shows > different behaviors). > > Is there a possibility that I have a corrupt setup on my machine? How to > double check? > > Thanks, > Mahmoud Ghoneim, PhD > Post Doctoral Research Scientist > Computer Science Department > School of Engineering and Applied Science > The George Washington University > > > On Wed, Feb 12, 2014 at 7:30 AM, Hieu Hoang <[email protected]> wrote: > >> Hi Mahmoud >> >> To post to the mailing list, please subscribe to it first. You can >> subscribe here >> http://mailman.mit.edu/mailman/listinfo/moses-support >> Also, if you need to post large datafiles, you should place the files >> somewhere for people to download it, and give people the URL. Don't post >> very large datafiles. I use google drive >> >> I've looked at your problem. I ran it without LMs and without all the >> command line arguments. >> $MOSES_DIR/bin/moses -config Sent15-FilteredModel/moses.ini >> -input-file sent151.txt >> $MOSES_DIR/bin/moses -config >> Sent15-FilteredModel/BinaryTables/moses.bin.ini -input-file sent151.txt >> >> They gave me exactly the same answer and model scores. So I'm not sure >> what the problem is. >> >> Remember - both lexical reordering models are used whenever a hypothesis >> is created, by either translation model. >> >> >> >> On 11 February 2014 19:15, <[email protected]> wrote: >> >>> As list administrator, your authorization is requested for the >>> following mailing list posting: >>> >>> List: [email protected] >>> From: [email protected] >>> Subject: I think moses has a bug when dealing with binarized >>> factored models >>> Reason: Post by non-member to a members-only list >>> >>> At your convenience, visit: >>> >>> http://mailman.mit.edu/mailman/admindb/moses-support >>> >>> to approve or deny the request. >>> >>> >>> ---------- Forwarded message ---------- >>> From: Mahmoud Ghoneim <[email protected]> >>> To: [email protected] >>> Cc: >>> Date: Tue, 11 Feb 2014 14:15:17 -0500 >>> Subject: I think moses has a bug when dealing with binarized factored >>> models >>> Hi moses-support team, >>> I have an Ar-En factored translation model (Ar: lexeme|lemma|POS to En: >>> lexeme|lemma|POS) with two translation paths (0-0 and 1,2-0). When I use >>> the text version of the model and the binarized version of the same exact >>> model, I get two totally different outputs!! >>> >>> I chose an input sentence and I generated the detailed verbose (v=3) >>> logging and compared the results for the two cases and it seems that the >>> decoder is not calculating the hypothesis's scores in a correct way when it >>> deals with binarized version of a factored model !! The problem is >>> generated due to considering all factors (rather than only the specific >>> decoding-path factors) when calculating the lexical-reordering scores >>> (explained in details below) >>> >>> Would you please review the attached files and confirm on my conclusions >>> or give me the proper explanation of this behavior? >>> >>> I am attaching a 'tgz' file >>> >> >> >>> containing the following: >>> 1- the input sentence (sent151.txt) >>> 2- the filtered text model (in subDirectory 'Sent15-FilteredModel') >>> 3- the binarized version of the filtered text model (in subDirectory >>> 'Sent15-FilteredModel/BinaryTables') >>> 4- the output translation using the text model (sent151.txt.out-mosesR1) >>> 5- the output translation using the binarized model >>> (sent151.txt.out-mosesR1-filtered-bin) >>> 6- the verbose logging files for both cases. () >>> 7- the commands used to run moses. (command.note) >>> >>> Examples of the problem: >>> - If you review the creation of hypothesis 6 from 0 (at line 2747 in >>> text model verbose and at line 2743 in the binarized model verbose), you >>> find moses considers the three factors while calculating the reordering >>> scores, (while it is supposed to consider factor0 only as this hypothesis >>> is form the first decoding-path 0-0) >>> - Also, if you review the creation of hypothesis 61 from 0 for the text >>> model which corresponds to hypothesis 64 for the binarized model, you will >>> find again that the lexical-reordering scores for decoding-path 0-0 is >>> included while this hypothesis is generated from the decoding-path 1,2-0!! >>> >>> Thanks in advance for your help, >>> Mahmoud Ghoneim, PhD >>> Post Doctoral Research Scientist >>> Computer Science Department >>> School of Engineering and Applied Science >>> The George Washington University >>> >>> >>> ---------- Forwarded message ---------- >>> From: [email protected] >>> To: >>> Cc: >>> Date: >>> Subject: confirm c4f798e58a24737bbfd9f76a5540c9d2c4b39cdd >>> If you reply to this message, keeping the Subject: header intact, >>> Mailman will discard the held message. Do this if the message is >>> spam. If you reply to this message and include an Approved: header >>> with the list password in it, the message will be approved for posting >>> to the list. The Approved: header can also appear in the first line >>> of the body of the reply. >>> >> >> >> >> -- >> Hieu Hoang >> Research Associate >> University of Edinburgh >> http://www.hoang.co.uk/hieu >> >> > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
