sure I am interested. Let me know the details and point me to the most probable code modules/parts to check. I'll check the current Moses in github now.
Thanks, Mahmoud Ghoneim, PhD Post Doctoral Research Scientist Computer Science Department School of Engineering and Applied Science The George Washington University On Wed, Feb 12, 2014 at 1:02 PM, Hieu Hoang <[email protected]> wrote: > You're right, there is a difference is model scores and output when using > the binary or memory phrase-table. > > This happens when I'm using Moses v1, but not if I'm using the current > Moses in github. > > Your lexical reordering models are fairly strange because they depend on a > source factor that only 1 phrase-table has. > > So you want to use only 1 lex. reordering model for each phrase-table? I'm > not sure if Moses is working as you intend. I'm not sure what it's doing > either! > > However, I think it's an interesting experiment so if you want to pursue > this and you're willing to do some coding, I can help you implement this. > > > > > > On 12 February 2014 14:28, Mahmoud Ghoneim <[email protected]> wrote: > >> Thanks Hieu for your reply. >> I changed my email address and I needed to register the new one. I did. >> >> I ran it on my machines without LMs and without all the command line >> arguments and again I got two different outputs! >> The attached file contains the two outputs. >> >> Also if "both lexical reordering models should be used whenever a >> hypothesis is created, by either translation model", then the text-based >> model is not working as supposed. Would you please review the verbose files >> I sent with the previous email. (I mentioned the exact lines that shows >> different behaviors). >> >> Is there a possibility that I have a corrupt setup on my machine? How to >> double check? >> >> Thanks, >> Mahmoud Ghoneim, PhD >> Post Doctoral Research Scientist >> Computer Science Department >> School of Engineering and Applied Science >> The George Washington University >> >> >> On Wed, Feb 12, 2014 at 7:30 AM, Hieu Hoang <[email protected]> wrote: >> >>> Hi Mahmoud >>> >>> To post to the mailing list, please subscribe to it first. You can >>> subscribe here >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> Also, if you need to post large datafiles, you should place the files >>> somewhere for people to download it, and give people the URL. Don't post >>> very large datafiles. I use google drive >>> >>> I've looked at your problem. I ran it without LMs and without all the >>> command line arguments. >>> $MOSES_DIR/bin/moses -config Sent15-FilteredModel/moses.ini >>> -input-file sent151.txt >>> $MOSES_DIR/bin/moses -config >>> Sent15-FilteredModel/BinaryTables/moses.bin.ini -input-file sent151.txt >>> >>> They gave me exactly the same answer and model scores. So I'm not sure >>> what the problem is. >>> >>> Remember - both lexical reordering models are used whenever a hypothesis >>> is created, by either translation model. >>> >>> >>> >>> On 11 February 2014 19:15, <[email protected]> wrote: >>> >>>> As list administrator, your authorization is requested for the >>>> following mailing list posting: >>>> >>>> List: [email protected] >>>> From: [email protected] >>>> Subject: I think moses has a bug when dealing with binarized >>>> factored models >>>> Reason: Post by non-member to a members-only list >>>> >>>> At your convenience, visit: >>>> >>>> http://mailman.mit.edu/mailman/admindb/moses-support >>>> >>>> to approve or deny the request. >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Mahmoud Ghoneim <[email protected]> >>>> To: [email protected] >>>> Cc: >>>> Date: Tue, 11 Feb 2014 14:15:17 -0500 >>>> Subject: I think moses has a bug when dealing with binarized factored >>>> models >>>> Hi moses-support team, >>>> I have an Ar-En factored translation model (Ar: lexeme|lemma|POS to En: >>>> lexeme|lemma|POS) with two translation paths (0-0 and 1,2-0). When I use >>>> the text version of the model and the binarized version of the same exact >>>> model, I get two totally different outputs!! >>>> >>>> I chose an input sentence and I generated the detailed verbose (v=3) >>>> logging and compared the results for the two cases and it seems that the >>>> decoder is not calculating the hypothesis's scores in a correct way when it >>>> deals with binarized version of a factored model !! The problem is >>>> generated due to considering all factors (rather than only the specific >>>> decoding-path factors) when calculating the lexical-reordering scores >>>> (explained in details below) >>>> >>>> Would you please review the attached files and confirm on my >>>> conclusions or give me the proper explanation of this behavior? >>>> >>>> I am attaching a 'tgz' file >>>> >>> >>> >>>> containing the following: >>>> 1- the input sentence (sent151.txt) >>>> 2- the filtered text model (in subDirectory 'Sent15-FilteredModel') >>>> 3- the binarized version of the filtered text model (in subDirectory >>>> 'Sent15-FilteredModel/BinaryTables') >>>> 4- the output translation using the text model (sent151.txt.out-mosesR1) >>>> 5- the output translation using the binarized model >>>> (sent151.txt.out-mosesR1-filtered-bin) >>>> 6- the verbose logging files for both cases. () >>>> 7- the commands used to run moses. (command.note) >>>> >>>> Examples of the problem: >>>> - If you review the creation of hypothesis 6 from 0 (at line 2747 in >>>> text model verbose and at line 2743 in the binarized model verbose), you >>>> find moses considers the three factors while calculating the reordering >>>> scores, (while it is supposed to consider factor0 only as this hypothesis >>>> is form the first decoding-path 0-0) >>>> - Also, if you review the creation of hypothesis 61 from 0 for the text >>>> model which corresponds to hypothesis 64 for the binarized model, you will >>>> find again that the lexical-reordering scores for decoding-path 0-0 is >>>> included while this hypothesis is generated from the decoding-path 1,2-0!! >>>> >>>> Thanks in advance for your help, >>>> Mahmoud Ghoneim, PhD >>>> Post Doctoral Research Scientist >>>> Computer Science Department >>>> School of Engineering and Applied Science >>>> The George Washington University >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: [email protected] >>>> To: >>>> Cc: >>>> Date: >>>> Subject: confirm c4f798e58a24737bbfd9f76a5540c9d2c4b39cdd >>>> If you reply to this message, keeping the Subject: header intact, >>>> Mailman will discard the held message. Do this if the message is >>>> spam. If you reply to this message and include an Approved: header >>>> with the list password in it, the message will be approved for posting >>>> to the list. The Approved: header can also appear in the first line >>>> of the body of the reply. >>>> >>> >>> >>> >>> -- >>> Hieu Hoang >>> Research Associate >>> University of Edinburgh >>> http://www.hoang.co.uk/hieu >>> >>> >> > > > -- > Hieu Hoang > Research Associate > University of Edinburgh > http://www.hoang.co.uk/hieu > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
