what we have is something that is very similar to the Google "bloomier filter" setup --ie a randomised LM, with the actual LM sharded across multiple machines. we have been working on making it faster and have some results here.
with any luck we will release this sometime early next year Miles On 17 November 2011 16:25, Christian Federmann <[email protected]> wrote: > Hi Peter, Hieu, all, > > my thesis stuff is rather outdated and likely not working with current Moses > code. > > As Hieu pointed out, the whole thing is problematic as networked requests > take much > longer than in-memory n-gram lookups. In the Dublin MT Marathon, Mark Fishel > and I > worked on optimal batching of LMServer requests and got pretty far; the > combination > of Miles' RandLM and such a batched, remote LM interface could be a nice > thing... > > Cheers, > Christian > > > > On Nov 17, 2011, at 2:53 PM, Hieu Hoang wrote: > >> hi peter >> >> i think christian federmann worked on the remote LM : >> >> https://www.google.com/search?hl=en&q=federmann+Very+large+language+models+for+machine+translation >> however, IMO, the decoder is lacking the infrastructure to do remote LM. >> >> to do it well, the decoder has to batch the LM calls to minimise second >> too many queries. Also, it has to make the calls asynchronously rather >> than wait for the LM query to complete. >> >> I'm not sure how far christian got but i suspect this is a major >> undertaking >> >> ps. your email to the mailing list went through fine. Why did you think >> it didn't? >> http://news.gmane.org/gmane.comp.nlp.moses.user >> >> On 17/11/2011 14:54, P.J. Berck wrote: >>> Hi, >>> >>> I was looking at the possibility to use a remote LM in moses, but I can't >>> find any documentation. >>> >>> I know about the "6 0 3 host:port" specification in moses.ini, but a naive >>> test just gives errors like "Your data contains<s> in a position other >>> than the first word." >>> >>> Is there some kind of protocol I need to implement? What kind of results >>> does moses expect? >>> >>> Thanks for pointers, >>> -peter >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
