the binary/on-disk phrase-tables created by
processPhraseTable
and
CreateOnDiskPt
work on slightly different ways so they won't give exactly the same
answer. However, the resulting BLEU scores should be similar.
The ways they are different:
1. CreateOnDiskPt only keeps the n-best p(e|f) for every source
phrase. n is the tableLimit argument when you run it. processPhraseTable
will keep all translation rules
2. The phrase-table created by processPhraseTable will sort and
prune the phrase-table during decoding according to the weighted scores.
The translation rules for each source phrase created by CreateOnDiskPt
will always be sorted and pruned according to p(e|f)
On 17/09/14 12:40, Eleftherios Avramidis wrote:
> Hi,
>
> the filtered phrase table is processed by filter-model-given-input.pl,
> with a parameter to use "processPhraseTable" for binarization
>
> best
> Lefteris
>
> PS: sorry for the delay, you were very fast, but Moses e-mails go to
> separate folder and I forgot to check it :-/
>
>
> On 15/09/14 21:14, Hieu Hoang wrote:
>> is the filtered phrase table also binarized with on-disk, or is it
>> text, or was it binarized with as PhraseDictionaryBinary?
>>
>> On 15/09/14 20:57, Eleftherios Avramidis wrote:
>>> Hi,
>>>
>>> I have trained a simple phrase-based system.
>>> First, I filtered the phrase table based on my test-set, and then I
>>> decoded the test-set (as typically EMS does).
>>> Secondly, I converted the original full phrase table to an On-Disk
>>> phrase table, and then sent the same sentences for decoding.
>>> My problem is that the sentences decoded with the on-disk full phrase
>>> table get different decoding than the ones from the filtered phrase
>>> table.
>>>
>>> I have checked that the moses ini weights, the language model, the
>>> re-ordering table, and the pre-processing scripts are the same.
>>>
>>> Does anybody know why this is happening?
>>>
>>> cheers
>>> Lefteris
>>>
>>>
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
