Re: [Mt-list] SMT Tools

[Chris Callison-Burch]

Dear Simon,

I would recommend that you use Philipp Koehn's Pharaoh decoder rather  
than the ISI ReWrite decoder.  Pharaoh uses phrase-based models of  
statistical machine translation rather than the older IBM-style word- 
based models.

You can download Pharaoh from
        http://www.isi.edu/licensed-sw/pharaoh/

and additional training scripts from
        http://www.iccs.informatics.ed.ac.uk/~pkoehn/training.tgz

There is a comprehensive manual that accompanies the program, as well  
as a conference paper that you can cite in in publications that use  
the program:

@inproceedings{ Koehn2004,
  author =      {Philipp Koehn},
  title =	{Pharaoh: A Beam Search Decoder for Phrase-Based  
Statistical Machine Translation Models},
  booktitle =   {Proceedings of AMTA},
  year =        {2004},
  url =	{http://www.iccs.informatics.ed.ac.uk/~pkoehn/publications/ 
pharaoh-amta2004.pdf}
}

Also, if you're interested in tinkering with the innards of a phrase- 
based decoder, I have started an open source project to create a  
phrase-based decoder which is in a fairly mature state.  Please e- 
mail me if you are interested.

Yours,
Chris Callison-Burch


On Nov 2, 2005, at 2:19 AM, Simon Zwarts wrote:

Hi,

I started working with some of the well known tools for statistical
machine translation and I have some questions about these tools.  I  
was
wondering if there is a technical mail list for smt tools where these
questions can be asked, or failing that whether anyone here would  
know the
answers. The questions concern at the moment the GIZA++ tool and  
the ISI
rewrite tool.

At the moment I have the following two questions:

ISI Rewrite system:

1) zerofert

I started following the instructions on
http://www.isi.edu/natural-language/software/decoder/manual.html
I tried to generate a zerofertility file with the tool  
rewrite.mkZeroFert.perl
Now this script needs two files: a vocabulary file which I have and
a .n4.final file (according to the comments in that file)

Now this file did not come out of a Giza++ run I did, nor is it  
mentioned on
the a webpage I found explaining the output of Giza++
(http://www-etud.iro.umontreal.ca/~demorali/traingizainfo.html)
Did I overlook a setting in giza++ or do I need another file?

2) distortion files

For calculation alignments one needs to use some probabilities for  
word order

Giza++ has:

D3Table          # distortion table for model 3
D4Table          # distortion table for model 4

Now, all the .d3.final files I manage to create with Giza++ have a  
"60" as
the third item, which means only distortion information is  
available for
sentence of length 60 if I understand it correctly. This can't be  
correct
but I have no clue what is going wrong The .D4.final file doesn't  
have any
description on the webpage mentioned, but the numbers look weird  
too. If I
assume the relative probability is the number mentioned there  
divided by
the sum I get this result for the head of cept:

...
-3 0.0170141
-2 0.0167512
-1 0.0132766
0 0.000601109
1 0.590987
2 0.142046
3 0.0633281
...

where I have a very small probability of a word staying in the same  
place,
which seems to be quite unlikely for my language pair (Dutch-English,
Europarl)

All other files generated by Giza++ seem to make sense and be correct.
_______________________________________________
Mt-list mailing list

_______________________________________________
Mt-list mailing list