Can you use Google n-grams through some API? How about word2vec ( https://code.google.com/p/word2vec/) ?
Best Regards, Ergun Ergun Biçici, CNGL, School of Computing, DCU, www.cngl.ie http://www.computing.dcu.ie/~ebicici/ On Sat, Apr 25, 2015 at 2:47 PM, Kenneth Heafield <[email protected]> wrote: > Hi, > > We know how to produce filtered models. The problem is StaticData > enforces one feature set per process. Lane could theoretically run > single-threaded and hack StaticData in between each sentence. The real > answer is that StaticData needs to die. > > Kenneth > > On 04/25/2015 07:19 AM, Ergun Bicici wrote: > > > > From man ngram: > > -limit-vocab > > Discard LM parameters on reading that do not pertain > > to the words specified in the vocabulary. The default is that > > words used in the LM are automatically added to the > > vocabulary. This option can be used to reduce the memory require‐ > > ments for large LMs that are going to be evaluated only on > > a small vocabulary subset. > > > > Best Regards, > > Ergun > > > > Ergun Biçici, CNGL, School of Computing, DCU, www.cngl.ie > > <http://www.cngl.ie> > > http://www.computing.dcu.ie/~ebicici/ > > > > > > On Fri, Apr 24, 2015 at 9:12 PM, Lane Schwartz <[email protected] > > <mailto:[email protected]>> wrote: > > > > To answer my own question... > > > > After talking with Hieu and Kenneth, it appears that the answer, at > > present, is no. But if anyone would be interested in working on this > > as an MT Marathon project, this would be great. > > > > On Fri, Apr 24, 2015 at 10:25 AM, Lane Schwartz <[email protected] > > <mailto:[email protected]>> wrote: > > > Does moses (and particularly EMS) have a mechanism to allow for > each > > > test sentence to have its own LM file that should be used when > > > translating just that sentence? > > > > > > This is in the context of taking a large LM and filtering it for a > > > single sentence. > > > > > > Thanks, > > > Lane > > > > > > > > -- > > When a place gets crowded enough to require ID's, social collapse is > not > > far away. It is time to go elsewhere. The best thing about space > > travel > > is that it made it possible to go elsewhere. > > -- R.A. Heinlein, "Time Enough For Love" > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
