Many thanks for this Kenneth.
Kind regards, Lee Ball Infrastructure Manager lee.b...@appliedlanguage.com Skype ID: lee.ball_appliedlanguage Tel: +44 (0)844 854 8945 Applied Language Solutions High quality language solutions delivered on time...with a smile! www.appliedlanguage.com Tel (UK): +44 (0)845 367 7000 Tel (US): +1 (800) 579-5010 Riverside Court, Huddersfield Road, Delph, Oldham, OL3 5FZ. UK Registered in the UK 5122429 Pride in everything we do | Respect everyone like a friend [image: An Environmentally Friendly Company]Think of the environment; please don't print this e-mail unless you really need to. [image: Fast Track 100 2009][image: Queens Award for Business] On 2 November 2010 17:56, Kenneth Heafield <mo...@kheafield.com> wrote: > My code is LGPL (same as Moses). kenlm/util/string_piece.hh and > kenlm/util/string_piece.cc come from Google under a permissive license > found in the comments at the beginning of these files. Austin Appleby, > author of kenlm/util/murmur_hash.cc, says "All code is released to the > public domain. For business purposes, Murmurhash is under the MIT license." > > On 11/02/10 13:44, Lee Ball (Applied Language) wrote: > > Hi, I've not seen it in this list but what licenses is KenLM distributed > > under? > > > > Kind regards, > > > > Lee Ball > > Infrastructure Manager > > lee.b...@appliedlanguage.com <mailto:lee.b...@appliedlanguage.com> > > Skype ID: lee.ball_appliedlanguage > > Tel: +44 (0)844 854 8945 > > > > Applied Language Solutions > > High quality language solutions delivered on time...with a smile! > > > > www.appliedlanguage.com <http://www.appliedlanguage.com/> > > Tel (UK): +44 (0)845 367 7000 > > Tel (US): +1 (800) 579-5010 > > > > Riverside Court, Huddersfield Road, Delph, Oldham, OL3 5FZ. UK > > Registered in the UK 5122429 > > > > Pride in everything we do | Respect everyone like a friend > > An Environmentally Friendly CompanyThink of the environment; please > > don't print this e-mail unless you really need to. > > > > Fast Track 100 2009Queens Award for Business > > > > > > > > > > On 19 October 2010 01:31, Kenneth Heafield <mo...@kheafield.com > > <mailto:mo...@kheafield.com>> wrote: > > > > Hi Moses, > > > > Introducing kenlm in Moses trunk. You no longer need to > > download a > > separate language model to use Moses; it's distributed with Moses and > > compiled in by default on UNIX. This is threadsafe language model > > inference code that returns the same probabilities as SRI (up to > > floating point rounding). It loads APRA files in 2/3 the time SRI > takes > > and uses less memory too. Using kenlm is simple: in your > [lmodel-file] > > section, change the first digit to 8. For example, > > > > "0 0 2 foo.arpa" changes to "8 0 2 foo.arpa" > > > > For even faster loading, use the binary format: > > > > kenlm/build_binary foo.arpa foo.binary > > > > then simply provide the binary filename in your moses.ini e.g. > > "8 0 2 foo.binary"; it auto detects binary files using magic bytes at > > the beginning. > > > > The code is ready for use and provides correct results. > > Inference is > > slower than it should be due to inefficiencies in the Moses-side > wrapper > > code (it does a vocab lookup for all 5 words every time). I'm > working > > on it and once this is done I'll post some benchmarks against SRI and > > IRST. The binary format is subject to change, but contains a version > > number so on very rare occasions after, new versions will tell you to > > rebuild your binary files. Windows is currently not supported (it > uses > > mmap) though I welcome contributions using #ifdef and > CreateFileMapping. > > > > Have fun and let me know about your experiences with it. > > > > "Ken" > > _______________________________________________ > > Moses-support mailing list > > Moses-support@mit.edu <mailto:Moses-support@mit.edu> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > > _______________________________________________ > > Moses-support mailing list > > Moses-support@mit.edu > > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
