Hi,
Here's a cheap server for fragment scoring.
socat TCP4-LISTEN:2000,fork EXEC:"bin/fragment lm/test.arpa"
Then in another terminal
socat TCP4-CONNECT:localhost:2000 STDIO <text
Now if you want to translate fragments instead then that's what Moses is
for, though keep in mind that it will always prepend <s> and append </s>
for translation.
Kenneth
On 05/26/14 02:04, Albert Llorens wrote:
> Thanks, Kenneth.
>
> Yes, I want to score sentence fragments. I want to use Moses for fragment
> translation, but only for frequent or probable fragments. I'll try what you
> suggest. Any chance the query could be done remotely, using mosesserver or
> anything else?
>
> Kind regards.
>
> Albert
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On
> Behalf Of Kenneth Heafield
> Sent: viernes, 23 de mayo de 2014 17:34
> To: [email protected]
> Subject: Re: [Moses-support] Get the probability of a given n-gram in a
> language model
>
> Hi,
>
> You can use bin/query on an ARPA or KenLM file. Then just type
> sentences at it (or use a file as stdin). By default it will assume you are
> scoring sentences. You can pass -n to not wrap in <s> and </s>.
>
> It appears that you are asking to score sentence fragments. The
> leading words will be scored using unigrams, bigrams, etc. from, say, a
> 5-gram model. If you are using Kneser-Ney, these lower-order probabilities
> (unigrams through 4-grams) are conditioned on having backed off to them. If
> you want accurate scores for sentence fragments, build a model of order 1,
> order 2, order 3, etc. then combine them using
>
> build_binary -r "1.arpa 2.arpa 3.arpa 4.arpa" 5.arpa 5.rest
>
> You can then use
>
> bin/fragment 5.rest <fragments
>
> to attain log10 frequencies. For more on this rant, read
>
> http://kheafield.com/professional/edinburgh/rest_paper.pdf
>
> Kenneth
>
> On 05/23/14 05:13, Albert Llorens wrote:
>> Hi,
>>
>>
>>
>> Is there a straightforward way I can ask Moses for the probability (or
>> the frequency) of a given n-gram in a given language model? If so, can
>> I do the query through mosesserver?
>>
>>
>>
>> Thanks.
>>
>>
>>
>> Kind regards.
>>
>>
>>
>> Albert
>>
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> [email protected]
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
